[cisco-voip] Trace files for CUCM/NTP problem

[Ed Leatherman]

Hi folks,

Our operations team updated the NTP service recently (infoblox), and
right after that happened, I started getting syslog errors per below
on two different CUCM 7 clusters, both of which use that NTP server.

ntpRunningStatus.sh: Primary node NTP server, OWP-PUB, is currently
inaccessible or down. Verify the network between the primary and
secondary nodes. Check the status of NTP on both the primary and
secondary nodes via CLI 'utils ntp status'. If the network is fine,
try restarting NTP using CLI 'utils ntp restart'.

Looking at the status on these servers, the pub looks OK but the subs show:
utils ntp status on all secondary nodes comes up with (example):
remote refid st t when poll reach delay offset jitter
==============================================================================
*127.127.1.0 LOCAL(0) 10 l 32 64 377 0.000 0.000 0.004
10.192.20.10 .STEP. 16 u 488 512 376 0.244 16.553 0.052

Restarting NTP on all nodes fixes the problem temporarily (NTP status
goes back to normal) but only for a short time.

The NTP logs don't show anything other than what appears to be the NTP
service restarting every 30 minutes.. is this normal?
11/16/2010 23:00:02
sd_ntp|*********************************************************|<LVL::Info>
11/16/2010 23:00:02 sd_ntp| Running sd_ntp. Process Id=12302
|<LVL::Info>
11/16/2010 23:00:02
sd_ntp|*********************************************************|<LVL::Info>
11/16/2010 23:00:02 sd_ntp||<LVL::Info>
11/16/2010 23:00:02 sd_ntp|[528] Command Line parameters: -list -s|<LVL::Info>
11/16/2010 23:00:02 sd_ntp|[585] The file /etc/ntp.conf exists|<LVL::Debug>
11/16/2010 23:00:02 sd_ntp|[421] /etc/ntp/drift file is not changed|<LVL::Debug>
11/16/2010 23:00:02 sd_ntp|[603] Listing all the servers|<LVL::Debug>
11/16/2010 23:00:02 sd_ntp|sd_ntp exitinng normally.|<LVL::Info>

In both clusters, the pub and most or all of the subs are on the same
VLAN and physical switch.

What other traces can I look at on CM to troubleshoot this? Anyone
know if there is a debug for the process that's generating my syslog
errors?

I want to make sure it's not an error on my end and hopefully have
some better information on whats broke before I go back to the
operations group. All the IOS routers using infoblox for NTP appear to
be working just fine, so they see no problems :)

Thanks in advance!

[Ed Leatherman]

Wes,

This is what one of the pubs looks like:
admin:utils ntp status
ntpd (pid 14883) is running...

remote refid st t when poll reach delay offset jitter
==============================================================================
127.127.1.0 LOCAL(0) 10 l 21 64 0 0.000 0.000 4000.00
10.0.1.132 .STEP. 16 u 77 64 0 0.000 0.000 4000.00
10.0.1.140 .STEP. 16 u 108 64 0 0.000 0.000 4000.00
10.0.1.148 .STEP. 16 u 117 64 0 0.000 0.000 4000.00

[Ed Leatherman]

I was about to say IOS devices are OK.. but i noticed the poll value
was 64 on the one I was reviewing... on a stable environment that
should be 1024 "steady-state" it sounds like. Some mischief is afoot.

Thanks for the NTP tips.

On Mon, Dec 13, 2010 at 7:12 PM, Jason Aarons (US)

Have you pointed a different router/switch to your NTP server? Are they
getting 16 as well? I recall a high offset/variation from clock can also
make it 16.
A IOS device initially polls every 64ms, as the NTP server and client are
better synced and there aren't dropped packets, this number increases to a
maximum of 1024
http://www.nil.si/ipcorner/BeOnTime/
http://www.cisco.com/en/US/products/sw/iosswrel/ps1818/products_tech_note09186a008015bb3a.shtml
“while the highest level (stratum 16) usually indicates that the clock is
not working or unaccessible”
Sent: Monday, December 13, 2010 6:57 PM
To: Wes Sisk
Cc: Cisco VOIP
Subject: Re: [cisco-voip] Trace files for CUCM/NTP problem
Ed,
CUCM is preferring the local clock, because your NTP reference has a Stratum
of 16!
10.192.20.10    .STEP.          16 u  488  512  376    0.244   16.553
0.052
Fix your NTP server 10.192.20.10 and you'll fix your CUCM.
-Burns
CSCsk70971    publisher NTP down if configured NTP down or unreliable
something on the network NTP source changed
now subscribers giving error that pub is unreliable
this is expected if pub cannot sync to NTP source. what changes did they
make? it is still a viable NTP source for hte publisher? if not, publisher
will use local clock which makes it an invalid source for all subs.
http://www.cisco.com/en/US/docs/voice_ip_comm/cucm/srnd/8x/netstruc.html#wpmkr1185636
/Wes
Hi folks,
Our operations team updated the NTP service recently (infoblox), and
right after that happened, I started getting syslog errors per below
on two different CUCM 7 clusters, both of which use that NTP server.
ntpRunningStatus.sh: Primary node NTP server, OWP-PUB, is currently
inaccessible or down. Verify the network between the primary and
secondary nodes.  Check the status of NTP on both the primary and
secondary nodes via CLI 'utils ntp status'.  If the network is fine,
try restarting NTP using CLI 'utils ntp restart'.
     remote           refid      st t when poll reach   delay   offset
jitter
==============================================================================
*127.127.1.0     LOCAL(0)        10 l   32   64  377    0.000    0.000
0.004
10.192.20.10    .STEP.          16 u  488  512  376    0.244   16.553
0.052
Restarting NTP on all nodes fixes the problem temporarily (NTP status
goes back to normal) but only for a short time.
The NTP logs don't show anything other than what appears to be the NTP
service restarting every 30 minutes.. is this normal?
11/16/2010 23:00:02
sd_ntp|*********************************************************|<LVL::Info>
11/16/2010 23:00:02 sd_ntp|          Running sd_ntp. Process Id=12302
               |<LVL::Info>
11/16/2010 23:00:02
sd_ntp|*********************************************************|<LVL::Info>
11/16/2010 23:00:02 sd_ntp||<LVL::Info>
11/16/2010 23:00:02 sd_ntp|[528] Command Line parameters: -list -s|<LVL::Info>
11/16/2010 23:00:02 sd_ntp|[585] The file /etc/ntp.conf exists|<LVL::Debug>
11/16/2010 23:00:02 sd_ntp|[421] /etc/ntp/drift file is not
changed|<LVL::Debug>
11/16/2010 23:00:02 sd_ntp|[603] Listing all the servers|<LVL::Debug>
11/16/2010 23:00:02 sd_ntp|sd_ntp exitinng normally.|<LVL::Info>
In both clusters, the pub and most or all of the subs are on the same
VLAN and physical switch.
What other traces can I look at on CM to troubleshoot this? Anyone
know if there is a debug for the process that's generating my syslog
errors?
I want to make sure it's not an error on my end and hopefully have
some better information on whats broke before I go back to the
operations group. All the IOS routers using infoblox for NTP appear to
be working just fine, so they see no problems :)
Thanks in advance!
_______________________________________________
cisco-voip mailing list
https://puck.nether.net/mailman/listinfo/cisco-voip
________________________________
Disclaimer: This e-mail communication and any attachments may contain
confidential and privileged information and is for use by the designated
addressee(s) named above only. If you are not the intended addressee, you
are hereby notified that you have received this communication in error and
that any use or reproduction of this email or its contents is strictly
prohibited and may be unlawful. If you have received this communication in
error, please notify us immediately by replying to this message and deleting
it from your computer. Thank you.
_______________________________________________
cisco-voip mailing list
https://puck.nether.net/mailman/listinfo/cisco-voip

[Jason Burns]

Ed,

[Antonio Soares]

Hello group,

Suppose you have CP-6911 Phones and you don't have Cisco Switches. How do
you configure the Voice Vlan ?

These Phones don't have a screen and the integrated IVR only allows you to
configure the basic network settings.


Thanks.

Regards,

Antonio Soares, CCIE #18473 (R&S/SP)
***@netcabo.pt

[Max Pierson]

Depends on what kind of switch you're using. On an HP Procurve, all you have
to do is 'tag' the port, otherwise known as trunking in Ciscoland and also
use the 'voice' keyword. The Procurve supports LLDP and so does your IP
phone.

ex.
2610(config)# vlan 1 name data
2610(config)# vlan 1 untagged 1-24
2610(config)# vlan 10 name voip
2610(config)# vlan 10 tagged 1-24
2610(config)# vlan 10 voice

As stated before, really depends on your vendor...

M

Post by Antonio Soares
Hello group,
Suppose you have CP-6911 Phones and you don't have Cisco Switches. How do
you configure the Voice Vlan ?
These Phones don't have a screen and the integrated IVR only allows you to
configure the basic network settings.
Thanks.
Regards,
Antonio Soares, CCIE #18473 (R&S/SP)
_______________________________________________
cisco-voip mailing list
https://puck.nether.net/mailman/listinfo/cisco-voip

[Antonio Soares]

In fact the switches are from HP and LLDP was tried but it didn’t work. So
my question about how to do it with non-Cisco, non-LLDP switches. Any
workarounds ?



I’m thinking about connecting the Phones to a Cisco Switch, let them learn
the Voice Vlan and then plug them to the other switches. Will it work ? Does
the Phone save the Voice Vlan information ?





Thanks.



Regards,



Antonio Soares, CCIE #18473 (R&S/SP)
<mailto:***@netcabo.pt> ***@netcabo.pt



From: Max Pierson [mailto:***@gmail.com]
Sent: quinta-feira, 16 de Dezembro de 2010 18:18
To: Antonio Soares
Cc: cisco-***@puck.nether.net
Subject: Re: [cisco-voip] CP-6911 and Voice Vlan



Depends on what kind of switch you're using. On an HP Procurve, all you have
to do is 'tag' the port, otherwise known as trunking in Ciscoland and also
use the 'voice' keyword. The Procurve supports LLDP and so does your IP
phone.



ex.

2610(config)# vlan 1 name data
2610(config)# vlan 1 untagged 1-24
2610(config)# vlan 10 name voip
2610(config)# vlan 10 tagged 1-24
2610(config)# vlan 10 voice



As stated before, really depends on your vendor...



M

On Thu, Dec 16, 2010 at 12:04 PM, Antonio Soares <***@netcabo.pt>
wrote:

Hello group,

Suppose you have CP-6911 Phones and you don't have Cisco Switches. How do
you configure the Voice Vlan ?

These Phones don't have a screen and the integrated IVR only allows you to
configure the basic network settings.


Thanks.

Regards,

Antonio Soares, CCIE #18473 (R&S/SP)
***@netcabo.pt

[Wes Sisk]

In that case the only option used to be to manually enter the vlan. I
have not tried this on the 69xx generation phones.

VVLAN is negotiated at every link transition. It should not be saved in
the manner described below.

/Wes

In fact the switches are from HP and LLDP was tried but it didn't
work. So my question about how to do it with non-Cisco, non-LLDP
switches. Any workarounds ?
I'm thinking about connecting the Phones to a Cisco Switch, let them
learn the Voice Vlan and then plug them to the other switches. Will it
work ? Does the Phone save the Voice Vlan information ?
Thanks.
Regards,
Antonio Soares, CCIE #18473 (R&S/SP)
*Sent:* quinta-feira, 16 de Dezembro de 2010 18:18
*To:* Antonio Soares
*Subject:* Re: [cisco-voip] CP-6911 and Voice Vlan
Depends on what kind of switch you're using. On an HP Procurve, all
you have to do is 'tag' the port, otherwise known as trunking in
Ciscoland and also use the 'voice' keyword. The Procurve supports LLDP
and so does your IP phone.
ex.
2610(config)# vlan 1 name data
2610(config)# vlan 1 untagged 1-24
2610(config)# vlan 10 name voip
2610(config)# vlan 10 tagged 1-24
2610(config)# vlan 10 voice
As stated before, really depends on your vendor...
M
Hello group,
Suppose you have CP-6911 Phones and you don't have Cisco Switches. How do
you configure the Voice Vlan ?
These Phones don't have a screen and the integrated IVR only allows you to
configure the basic network settings.
Thanks.
Regards,
Antonio Soares, CCIE #18473 (R&S/SP)
_______________________________________________
cisco-voip mailing list
https://puck.nether.net/mailman/listinfo/cisco-voip
------------------------------------------------------------------------
_______________________________________________
cisco-voip mailing list
https://puck.nether.net/mailman/listinfo/cisco-voip

[Antonio Soares]

The problem is that the 6911 doesn’t have a screen and the IVR doesn’t seem
to have the option to set the voice vlan.



Thanks.



Regards,



Antonio Soares, CCIE #18473 (R&S/SP)
<mailto:***@netcabo.pt> ***@netcabo.pt



From: Wes Sisk [mailto:***@cisco.com]
Sent: quinta-feira, 16 de Dezembro de 2010 19:14
To: Antonio Soares
Cc: 'Max Pierson'; cisco-***@puck.nether.net
Subject: Re: [cisco-voip] CP-6911 and Voice Vlan



In that case the only option used to be to manually enter the vlan. I have
not tried this on the 69xx generation phones.

VVLAN is negotiated at every link transition. It should not be saved in the
manner described below.

/Wes

Antonio Soares wrote:

In fact the switches are from HP and LLDP was tried but it didn’t work. So
my question about how to do it with non-Cisco, non-LLDP switches. Any
workarounds ?



I’m thinking about connecting the Phones to a Cisco Switch, let them learn
the Voice Vlan and then plug them to the other switches. Will it work ? Does
the Phone save the Voice Vlan information ?





Thanks.



Regards,



Antonio Soares, CCIE #18473 (R&S/SP)
***@netcabo.pt



From: Max Pierson [mailto:***@gmail.com]
Sent: quinta-feira, 16 de Dezembro de 2010 18:18
To: Antonio Soares
Cc: cisco-***@puck.nether.net
Subject: Re: [cisco-voip] CP-6911 and Voice Vlan



Depends on what kind of switch you're using. On an HP Procurve, all you have
to do is 'tag' the port, otherwise known as trunking in Ciscoland and also
use the 'voice' keyword. The Procurve supports LLDP and so does your IP
phone.



ex.

2610(config)# vlan 1 name data
2610(config)# vlan 1 untagged 1-24
2610(config)# vlan 10 name voip
2610(config)# vlan 10 tagged 1-24
2610(config)# vlan 10 voice



As stated before, really depends on your vendor...



M

On Thu, Dec 16, 2010 at 12:04 PM, Antonio Soares <***@netcabo.pt>
wrote:

Hello group,

Suppose you have CP-6911 Phones and you don't have Cisco Switches. How do
you configure the Voice Vlan ?

These Phones don't have a screen and the integrated IVR only allows you to
configure the basic network settings.


Thanks.

Regards,

Antonio Soares, CCIE #18473 (R&S/SP)
***@netcabo.pt

_______________________________________________
cisco-voip mailing list
cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip





_____

[Wes Sisk]

Searching around and taking a step back LLDP should work. What load are
you running on the phones? Looks like there is a known issue with the
6911's getting vvlan from LLDP in
CSCtb78295 6911 LLDP: would not send right voice vlan information

which appears to be fixed in 9.0(2). Defect will be visible on
Cisco.com in 24 hours. I don't see any way so far to set vlan via IVR
so LLDP troubleshooting may be in order.

/Wes

The problem is that the 6911 doesn't have a screen and the IVR doesn't
seem to have the option to set the voice vlan.
Thanks.
Regards,
Antonio Soares, CCIE #18473 (R&S/SP)
*Sent:* quinta-feira, 16 de Dezembro de 2010 19:14
*To:* Antonio Soares
*Subject:* Re: [cisco-voip] CP-6911 and Voice Vlan
In that case the only option used to be to manually enter the vlan. I
have not tried this on the 69xx generation phones.
VVLAN is negotiated at every link transition. It should not be saved
in the manner described below.
/Wes
In fact the switches are from HP and LLDP was tried but it didn't
work. So my question about how to do it with non-Cisco, non-LLDP
switches. Any workarounds ?
I'm thinking about connecting the Phones to a Cisco Switch, let them
learn the Voice Vlan and then plug them to the other switches. Will it
work ? Does the Phone save the Voice Vlan information ?
Thanks.
Regards,
Antonio Soares, CCIE #18473 (R&S/SP)
*Sent:* quinta-feira, 16 de Dezembro de 2010 18:18
*To:* Antonio Soares
*Subject:* Re: [cisco-voip] CP-6911 and Voice Vlan
Depends on what kind of switch you're using. On an HP Procurve, all
you have to do is 'tag' the port, otherwise known as trunking in
Ciscoland and also use the 'voice' keyword. The Procurve supports LLDP
and so does your IP phone.
ex.
2610(config)# vlan 1 name data
2610(config)# vlan 1 untagged 1-24
2610(config)# vlan 10 name voip
2610(config)# vlan 10 tagged 1-24
2610(config)# vlan 10 voice
As stated before, really depends on your vendor...
M
Hello group,
Suppose you have CP-6911 Phones and you don't have Cisco Switches. How do
you configure the Voice Vlan ?
These Phones don't have a screen and the integrated IVR only allows you to
configure the basic network settings.
Thanks.
Regards,
Antonio Soares, CCIE #18473 (R&S/SP)
_______________________________________________
cisco-voip mailing list
https://puck.nether.net/mailman/listinfo/cisco-voip
------------------------------------------------------------------------
_______________________________________________
cisco-voip mailing list
https://puck.nether.net/mailman/listinfo/cisco-voip

[Antonio Soares]

I’m not 100% sure but I think the phones are running 9.0(2). I will check
that bug when it becomes available.



Thanks.



Regards,



Antonio Soares, CCIE #18473 (R&S/SP)
<mailto:***@netcabo.pt> ***@netcabo.pt



From: Wes Sisk [mailto:***@cisco.com]
Sent: quinta-feira, 16 de Dezembro de 2010 20:24
To: Antonio Soares
Cc: 'Max Pierson'; cisco-***@puck.nether.net
Subject: Re: [cisco-voip] CP-6911 and Voice Vlan



Searching around and taking a step back LLDP should work. What load are you
running on the phones? Looks like there is a known issue with the 6911's
getting vvlan from LLDP in
CSCtb78295 6911 LLDP: would not send right voice vlan information

which appears to be fixed in 9.0(2). Defect will be visible on Cisco.com in
24 hours. I don't see any way so far to set vlan via IVR so LLDP
troubleshooting may be in order.

/Wes

Antonio Soares wrote:

The problem is that the 6911 doesn’t have a screen and the IVR doesn’t seem
to have the option to set the voice vlan.



Thanks.



Regards,



Antonio Soares, CCIE #18473 (R&S/SP)
***@netcabo.pt



From: Wes Sisk [mailto:***@cisco.com]
Sent: quinta-feira, 16 de Dezembro de 2010 19:14
To: Antonio Soares
Cc: 'Max Pierson'; cisco-***@puck.nether.net
Subject: Re: [cisco-voip] CP-6911 and Voice Vlan



In that case the only option used to be to manually enter the vlan. I have
not tried this on the 69xx generation phones.

VVLAN is negotiated at every link transition. It should not be saved in the
manner described below.

/Wes

Antonio Soares wrote:

In fact the switches are from HP and LLDP was tried but it didn’t work. So
my question about how to do it with non-Cisco, non-LLDP switches. Any
workarounds ?



I’m thinking about connecting the Phones to a Cisco Switch, let them learn
the Voice Vlan and then plug them to the other switches. Will it work ? Does
the Phone save the Voice Vlan information ?





Thanks.



Regards,



Antonio Soares, CCIE #18473 (R&S/SP)
***@netcabo.pt



From: Max Pierson [mailto:***@gmail.com]
Sent: quinta-feira, 16 de Dezembro de 2010 18:18
To: Antonio Soares
Cc: cisco-***@puck.nether.net
Subject: Re: [cisco-voip] CP-6911 and Voice Vlan



Depends on what kind of switch you're using. On an HP Procurve, all you have
to do is 'tag' the port, otherwise known as trunking in Ciscoland and also
use the 'voice' keyword. The Procurve supports LLDP and so does your IP
phone.



ex.

2610(config)# vlan 1 name data
2610(config)# vlan 1 untagged 1-24
2610(config)# vlan 10 name voip
2610(config)# vlan 10 tagged 1-24
2610(config)# vlan 10 voice



As stated before, really depends on your vendor...



M

On Thu, Dec 16, 2010 at 12:04 PM, Antonio Soares <***@netcabo.pt>
wrote:

Hello group,

Suppose you have CP-6911 Phones and you don't have Cisco Switches. How do
you configure the Voice Vlan ?

These Phones don't have a screen and the integrated IVR only allows you to
configure the basic network settings.


Thanks.

Regards,

Antonio Soares, CCIE #18473 (R&S/SP)
***@netcabo.pt

_______________________________________________
cisco-voip mailing list
cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip





_____

[Max Pierson]

Strange indeed. LLDP seems to work fine with the 79xx series on
8.(something) firmware. Since I have no experience with that model handset,
I'd do a packet capture and see whats going on with LLDP. Not having the
screen on the phone hurts when troubleshooting. If I recall, when the phone
boots, CDP/LLDP does the vlan magic and if it's not being 'tagged' by the
switch, it uses vlan 1, so I don't believe the phone retains the VLAN info.

M

Post by Antonio Soares
In fact the switches are from HP and LLDP was tried but it didn’t work. So
my question about how to do it with non-Cisco, non-LLDP switches. Any
workarounds ?
I’m thinking about connecting the Phones to a Cisco Switch, let them learn
the Voice Vlan and then plug them to the other switches. Will it work ? Does
the Phone save the Voice Vlan information ?
Thanks.
Regards,
Antonio Soares, CCIE #18473 (R&S/SP)
*Sent:* quinta-feira, 16 de Dezembro de 2010 18:18
*To:* Antonio Soares
*Subject:* Re: [cisco-voip] CP-6911 and Voice Vlan
Depends on what kind of switch you're using. On an HP Procurve, all you
have to do is 'tag' the port, otherwise known as trunking in Ciscoland and
also use the 'voice' keyword. The Procurve supports LLDP and so does your IP
phone.
ex.
2610(config)# vlan 1 name data
2610(config)# vlan 1 untagged 1-24
2610(config)# vlan 10 name voip
2610(config)# vlan 10 tagged 1-24
2610(config)# vlan 10 voice
As stated before, really depends on your vendor...
M
Hello group,
Suppose you have CP-6911 Phones and you don't have Cisco Switches. How do
you configure the Voice Vlan ?
These Phones don't have a screen and the integrated IVR only allows you to
configure the basic network settings.
Thanks.
Regards,
Antonio Soares, CCIE #18473 (R&S/SP)
_______________________________________________
cisco-voip mailing list
https://puck.nether.net/mailman/listinfo/cisco-voip

[Brian Schultz]

It isn't clear from the admin guide if there is a way to manually set the
VLAN through the IVR. Have you tried pressing *, #, 0 to check the settings
via the IVR?

http://www.cisco.com/en/US/docs/voice_ip_comm/cuipph/6901_6911/8_5/english/admin_guide/6911net.html

Thanks,
Brian

Post by Antonio Soares
In fact the switches are from HP and LLDP was tried but it didn’t work. So
my question about how to do it with non-Cisco, non-LLDP switches. Any
workarounds ?
I’m thinking about connecting the Phones to a Cisco Switch, let them learn
the Voice Vlan and then plug them to the other switches. Will it work ? Does
the Phone save the Voice Vlan information ?
Thanks.
Regards,
Antonio Soares, CCIE #18473 (R&S/SP)
*Sent:* quinta-feira, 16 de Dezembro de 2010 18:18
*To:* Antonio Soares
*Subject:* Re: [cisco-voip] CP-6911 and Voice Vlan
Depends on what kind of switch you're using. On an HP Procurve, all you
have to do is 'tag' the port, otherwise known as trunking in Ciscoland and
also use the 'voice' keyword. The Procurve supports LLDP and so does your IP
phone.
ex.
2610(config)# vlan 1 name data
2610(config)# vlan 1 untagged 1-24
2610(config)# vlan 10 name voip
2610(config)# vlan 10 tagged 1-24
2610(config)# vlan 10 voice
As stated before, really depends on your vendor...
M
Hello group,
Suppose you have CP-6911 Phones and you don't have Cisco Switches. How do
you configure the Voice Vlan ?
These Phones don't have a screen and the integrated IVR only allows you to
configure the basic network settings.
Thanks.
Regards,
Antonio Soares, CCIE #18473 (R&S/SP)
_______________________________________________
cisco-voip mailing list
https://puck.nether.net/mailman/listinfo/cisco-voip
_______________________________________________
cisco-voip mailing list
https://puck.nether.net/mailman/listinfo/cisco-voip