Telium Support Group

Since the Peerlink protocol verion has changed, the peers will not be able to talk to each other (over Peerlink) until both peers are upgraded. So if both peers are online at the same time they will not be able to communicate – and both peers will try to take over as active. Consequently, your upgrade procedure must ensure both peers are NOT online at the same time.

Since the license version has changed, you will need to request new licenses from Telium. To avoid bringing down the entire cluster you should upgrade and re-license one peer at a time.

The overall steps to performing such an upgrade are:

1. Upgrade A
   1. Stop HAAst on peer A, wait for stop
   2. Run the install_files/updatefiles.sh script from the newly downloaded package
   3. Unplug the network connection from A
   4. Restart HAAst on peer A
   5. Request and apply new license to A if required, then restart A
2. Switchover
   1. Stop HAAst on peer B, wait for stop
   2. Replug the network connection to A, ensure A takes over as active
3. Upgrade B
   1. Run the install_files/updatefiles.sh script from the newly downloaded package
   2. Restart HAAst on peer B, ensure cluster forms
   3. Request and apply new license to B if required, then restart B
4. Fallback to preferred peer (optional)
   1. Use the telnet/web interface to make the preferred peer active. (Or wait for automatic fallback during the maintenance window if enabled in the haast.conf file)

The answer depends on the location of your two PBX’s. If the PBX’s are located in the same data center (i.e. using the same external IP address), then no change is necessary as they will connect to the same IP address. If you need to modify your firewall/router internally to direct traffic to the active peer then see the answer to the question on locating the PBX for internal phones. On the other hand, if the PBX’s are located in different data centers (i.e. accessible using different public IP addresses) then your options are:

1. SRV records: Assuming your user agents (phone sets) support SRV records (which most do), then you should create SRV records for your two PBX’s. Most user agents will perform a DNS lookup for SRV records to find available PBX’s, and try them in order of priority until they successfully register with a PBX. For example, if you have PBX’s located in data centers dc1 and dc2, then create to DNS entries (in your public DNS server) as follows:

   
type=srv
name=_sip._udp.mydomain.com
priority= 10
weight=0
port=5060
hostname=dc1.mydomain.com

   
   and
   
type=srv
name=_sip._udp.mydomain.com
priority= 20
weight=0
port=5060
hostname=dc2.mydomain.com

2. DNS update: Use the pre/post asterisk start/stop event handlers to update a public DNS entry. Be sure to set the TTL value low enough that phones will lookup the new IP in a reasonable timeframe.
3. MPLS: If you use MPLS then you can simply move the label (to move IP between routers of your two DC’s). We don’t provide any further detail on this option (i.e. if you don’t understand how to do this with MPLS, then there’s too much to explain in one post)

Note: Using SRV records or DNS entries makes it easy for users with softphones to move on and off LAN and resume a PBX connection without manual intervention.

You are welcome to use rsync,NFS,samba, etc. in your cluster. However, we generally recommend keeping data on each peer and allowing HAAst to control all synchronization, and here’s why:

1. HAAst only synchronizes data between peers if peers have passed a health check. That means if one node is failing and starts to accidentally corrupt data, it will not be copied to the other peer! Tools like rsync, NFS, DRBD, etc. will immediately share/mirror all data including, corrupt data.
2. By allowing HAAst to control synchronization, the HAAst event handler system will allow you to customize inbound data following a synchronization (e.g. update trunk information, modify the dialplan, customize TFTP files for the local network, etc)

You should not place databases on any block level sharing device (NFS/SAMBA), or do block level mirroring (DRBD,iSCSI), as corruption by one peer will destroy the database for the other peer! Even worse, a failure midway through a write will corrupt both peers! Note that HAAst performs SQL transactions (not block level access) for database synchronization, so even if a peer fails midway through a database write neither peer will be left with an invalid database state.

The one exception to this rule is if you need to archive a high volume of files, or very large files, that are written once and thereafter only read. A perfect example of this is call center call recordings or logs. A call center can easily generate gigabytes of recordings every minute, to be referenced in the future in case of dispute or for quality assurance. Since these are large files written once and then archived, they are the perfect example of data that should be written to a server share, common iSCSI device, etc. It would not make sense to generate the high network load and disk load required to continually create a second copy of this data.

You’re very welcome. It was a great project, and I’m glad we were your partner for this important project.

When the cluster reassembles HAAst will discover 2 peers active (called ‘dual-active contention’). HAAst will then try to pick the peer with the lowest likelihood of long-term success (probability of staying active) and demote it. The determination of which peer is least likely to succeed considers:

• Which peer caused the previous failover
• How many failures has each peer had
• How long has each peer been running
• What was the last health score of each peer
• And more…

This works well when the peers are configured as equals (primary/primary), which implies that the cluster would be happy with either peer running. However, when the peers are not configured as equals (primary/backup), the determination of which peer should demote may result in the backup server remaining active. This describes the situation you encountered, and this is normal behavior (by design).

As of version 2.3.2.14 the administrator can override the demotion decision, ignoring the criteria listed above. If the ‘autodemote’ key is set to true in the [backuprole] stanza of either peer, then HAAst will always demote that peer.

No. Split brain usually refers to a mirrored file system (e.g.: DRBD) in which the two sides have gone out of sync. Proper recovery from split brain usually involves manually choosing which files to keep, one file at a time (or risk losing all data from one side if you blindly accept once site as correct). Since other products use block level mirroring, an interruption in the mirroring can leave files/databases in an inconsistent state and prevent Asterisk from starting or operating correctly.

HAAst on the other hand does not use a mirrored file system. In fact HAAst is the only HA system for Asterisk that does not use block level mirroring. HAAst synchronizes files/directories/databases/tables from Active to Standby only, and only when the peers are both confirmed healthy. Files use differential analysis and compression to send only changes, and databases use SQL level transactions to ensure databases are always in a consistent state.

With HAAst data is not sent if a node is detected to be in an unhealthy state, so potentially damaged files/databases are never sent to the other peer. Once a node recovers from a failure, the data from the healthy node will be sent to the recovered node to bring it back into sync. You will never have a split brain scenario with HAAst.

If that didn’t answer your question please provide more details.

The answer to this question depends on the specific product you are referring to. However, here are some points to consider:

1. Most manufacturers require that only a single copy of a license be in use at one time. Since HAAst keeps only one Asterisk peer active at a time, you should be ok keeping the same license on both computers peers at once and remain compliant.
2. Some manufacturers require that only a single copy of a license be installed at once. Using the pre-start & post-stop event handlers you can move the licenses into place before Asterisk starts, and out of place after Asterisk stops, and remain compliant.
3. A few manufacturers require that you purchase a license for each host regardless of whether it is used for standby/clustering/testing/development/etc. However, most manufacturers will offer such a license at no/minimal charge.

When you evaluate licensed products for purchase, the above should be one of your evaluation criteria. Most manufacturers are reasonable and recognize that charging for a standby license is excessive – and only antagonizes their customers. Still…we know of one manufacturer in particular that takes a rather hostile view towards their customers and expects a full price license even for standby servers. In some cases you may find open source alternatives that help alleviate unfair license restrictions (for example see: https://telium.io/topic/g729-licenses-invalidated-after-cluster-failover/)

Note that SecAst (Security for Asterisk) includes a second no-charge license for use in a HAAst cluster; so one license covers 2 peer installations.

Please note that the above is not legal advice – you should check with your legal counsel for all matters of license compliance, etc.

• This reply was modified 4 years, 8 months ago by WebMaster.

Support hours available to you depend on the edition of HAAst you are using, whether you have an active maintenance agreement, and whether you have a 24/7 support contract:

1. All users: Telium offers self-serve forums for users of non-commercial and commercial editions of its products. These forums are monitored by Telium support and questions are answered on a best effort basis.
2. Maintenance Agreements: Telium offers e-mail support during business hours for users with an active maintenance agreement. Telium business hours are 9am-5pm Eastern time, Monday to Friday (except holidays). Typical response times are under an hour, and typical problem resolution is less than 30 minutes. Telium may also respond up to 11pm Eastern time on weekdays, and anytime on weekends – depending on whether an on-call support rep was already activated for a 24/7 Support Contract (and answers outstanding questions before wrapping up).
3. 24/7 Support Contract: Telium offers 24/7 support for users who have a 24/7 support agreement in place. Support is offered by phone / skype / teamviewer / email / chat / etc, and includes rapid callback from the account manager / support lead / on-call technician (depending on time of day). Typical response times are under 20 minutes, and typical problem resolution is less than 30 minutes. Note that remote desktop connections are only for simple viewing/assistance, not complex work or complete installations.

So if it’s 2am Pacific time and you don’t have a 24/7 support contract in place then your only course for immediate support is the online forums.

Please note: If you are doing an installation and would like to book assistance from Telium professional services outside of normal business hours, we would be happy to work with you anytime from 8am to 2am Eastern time to perform your installation.