Forum Replies Created
-
AuthorPosts
-
in reply to: All HA for PBX based on open source? #6755
Telium wrote HAAst from top to bottom in C++, and we agree with your Linux admins that the open source packages are not well suited to application level HA. The open source packages are great if you want to create a HA print server, file server, etc. However, they have no understanding of, or visibility into, the PBX, the environment, trunks, etc. The open source packages work at the OS level, not the application level and as such don’t create an HA PBX which can survive real world failures. For example, a route becoming unavailable at the ITSP, the PBX running out of file handles, a data center router going down, etc. would all trigger a failover with HAAst, but open source packages would likely leave you with a non-functional PBX.
Aside from HAAst, every other HA solution for PBX’s is based on essentially the same open source packages. HAAst does not use open source packages for any detection, heartbeat, failover control, etc. But, you are welcome to use open sources packages with HAAst if you like. (There are some things open source does well and we don’t want to reinvent the wheel). Although the HAAst product tabs do a good job explaining the differences between HAAst and the open source / commercial products but here are a few highlights:
Heartbeat & Health: The open source heartbeat/health package does not take into account the health of Asterisk, status of trunk & route availability, available file handles, available memory, etc. latency between devices, calls successfully bridging, etc. Open source packages do dead/alive detection of the box/process (and are not Asterisk operations aware). HAAst has its own proprietary heartbeat and health detection, written exclusively to monitor and detect Asterisk PBX’s and their environments. By default HAAst detects 18 different factors which degrade node health (including talking directly to Asterisk through the AMI to assess health), and can also use an unlimited number of customizable sensors.
Synchronization: Open source packages like DRBD (or other shared disk solutions like NFS/Samba/iSCSI/Corosync/Rsync/etc.) put your data at risk since file corruption by one failing peer immediately corrupts files of the other peer. With those open source solutions a failing process on one peer may destroy your entire cluster. As well, network loss during data sharing may leave your files corrupt and SQL databases in invalid states (the database might not start with corrupt tables, and Asterisk might not start with corrupt config files). HAAst synchronizes data between peers, but only if the peers are healthy. HAAst synchronizes databases (MySQL/PostgreSQL/SQLite) at the SQL transaction level, so you are never left with corrupt tables (and a PBX that won’t start). And to top it off, HAAst can maintain snapshots of healthy critical files, stepping backwards through previous snapshots to find a system state that allows the PBX to start after failure.
Security: HAAst has an encrypted link between peers so there is no risk of Man In The Middle attack, and no risk of hackers gaining control of the PBX. Open source products have well published and unencrypted protocols that are easy to tap into, and a novice/script kiddie can bring down the cluster using this information.
Other: There are lots of other functional differences as well – have a look at the features tab of the HAAst web pages for an overview of what HAAst does, and then look at the comparison tab to see what generic tools can’t do.
Performance: The bottom line is: how do these HA solutions perform in real life. There’s a reason that emergency service gateways, hospitals, airline call centers, etc. choose HAAst. From complete detection, to avoiding false positive failovers, to speed of failover, to moving resources, etc. nothing comes close to HAAst.
All of the open source packages are wonderful products in their own right, each with a specific purpose. If you spend enough time adding your own code on top of these then you can start to add application level HA functionality – but its up to you to code it. After 10 years of continuous development we have created a very sophisticated product which can detect and recover gracefully from an enormous number of failure scenarios, building our own heartbeat/synchronization/health detection/etc tailored to telephony environments.
So unlike some other HA ‘solutions’, HAAst is not a collection of open source packages relabeled as an application level HA product. Check out the HAAst web page tabs to see why HAAst is the only solution for hospitals, police/fire/911 call centers, mission critical call centers,etc. So your Linux admins are right – open source packages are NOT suitable for application level HA (as would be the case with a PBX), and that’s why HAAst is the choice of large call centers/fire departments/emergency service gateways/etc.
in reply to: Send SNMP trap on failover #6754You can have HAAst issue SNMP traps using HAAst’s built-in event handler system. The event handlers are simply executable files (binary / BASH / etc) or symlinks placed in the directory:
/usr/local/haast/eventsThe event handlers are named to reflect when they are triggered. The files you most likely want to create are:
- asterisk.start.pre
- asterisk.stop.post
The event handler (files) are launched automatically by HAAst – there is nothing you need to do to execute them. For example, the “asterisk.start.pre” is launched by HAAst before Asterisk is started upon node promotion. If you wanted the event handler to run immediately after Asterisk is started, create a file called “asterisk.start.post”.
In your case create these two files with 550 permissions in the above folder. The content of the file would be similar to:
#!/bin/bash
# Issue SNMP v3 trap
snmptrap -v -e -u -a -A -x -X
logger “SNMP trap issued for demotion of local peer”There are also numerous environment variables set by HAAst before executing the event handlers; so if you want information related to the peer/failover/etc you can use these environment variables in your BASH script. (See the installation guide for more information on the environment variables, or use the set command in an event handler to redirect all environment variables to a temporary file for further examination).
The snmptrap commands may require a change to system configuration files, but that would be specific to your setup. If you have configured other traps then this is likely already done.
Please note that use of SNMP is an advanced topic, and its configuration requires advanced Linux admin skills.
in reply to: Where can I find an installation manual #6599Look in the /docs folder of the package you downloaded. In there you will find a PDF document called Detailed_Installation_Guide.pdf which will take you through all of the steps involved in installation (getting the program installed), and key steps of configuration (making the program work the way you want).
For example, if you copy the text below and place it into the file /etc/xdg/telium/haast.conf.d/network_cable.sensors.conf then you can test the sensors graph on the GUI. This sensors checks if the network connection on ‘eth1’ is down. If the connection is down the health score increases by 5 points. If the connection is up the health score remains unchanged for that cycle. (i.e. 0 health points).
If HAAst can’t read the NIC, then an error score of 20 is used. If HAAst can talk to the NIC but not extract a state then the warning score of 10 is used. If the sensors reports 0, then the cumulative score for this sensors is reset to 0. This sensor runs every 5 seconds.
So restart HAAst with this sensor in place and open up the graph on the HAAst GUI. Unlpug the network cable and watch your score, then replug the cable and see it reset to 0. If you leave the cable unplugged long enough and the health score reaches the critical level, then your cluster will fail-over to the peer.
; Test to ensure a network cable is plugged into the NIC, and that the cable is live (i.e. other end is plugged in)
network-cable/description=Cable plugged in
network-cable/type=nic
network-cable/input=state
network-cable/parameters=state:ethernet | interface:eth1
network-cable/scoring= =up:0 | :5
network-cable/warningscore=10
network-cable/errorscore=20
network-cable/resetcumulativescore=0
network-cable/interval=5
network-cable/debug=false
-
This reply was modified 5 years, 7 months ago by
WebMaster.
in reply to: iptables+fireHOL not blocking IP’s #6749Although this topic is a year old, it continues to get a lot of traffic. So, I would like to reiterate a key point mentioned above (in case you missed it):
You should not block IP’s at the PBX. (Unless this is test/home system). Commercial environments should block attackers at the firewall. SecAst has the ability to add IP’s to ACL’s/lists on your router / firewall. You really should use this feature!
in reply to: Power on/off VHF radios on active/standby server #6748Yes HAAst can manage the power for other devices like amplifiers, radio equipment, etc.
Using the “event hander” system in HAAst, HAAst can control IP controlled power strips and UPS’s. By doing so HAAst can turn on power to these devices when switching to active, and power off these devices when switching to standby. For example, customers have used this Synaccess NP-05B IP Managed Remote Power Switch/Strip:
You will find 120v, 240v, 15A/20A, and more models depending on what you want to switch. APC and Tripplite sell a variety of these (called switched PDU’s) so I’m sure you can find one that meets your power requirements.
HAAst can even serialize power up (to avoid current draw spikes) as the devices are turned on. Please contact the support group (support@telium.io) for further assistance.
HAAst includes approximately 18 “internal sensors” which are used to determine if the PBX and it’s environment are within normal parameters, and allow the PBX to properly offer telephony services. These sensors are used to assess whether or not the node can keep operating or must demote and let the peer take over. This is a binary decision (yes/no) determined by HAAst. This assessment and determination are made behind the scenes, and have nothing to do with the sensors graph in the GUI.
Aside from the internal sensors, HAAst also has the ability to monitor optional “external sensors”. These sensor are defined by you (the administrator) and can sense just about anything. For example, clients have used external sensors to measure:
- Performance of a network path: Counting pings lost from source IP to destination IP. This can be used to measure route quality, or even something as simple as sense an unplugged network cable.
- Calls in progress: Some large call centers never have less than X calls per second coming in from their carrier. If this drops to 0 it might mean an upstream carrier issue, 800 number failure, etc.
- Data center environmental temperature: If the data center room temperature exceeds X degrees HAAst can initiate a fail over.
- Custom: Anything from a contact closure (big red fail over button), to a local CRM systems being down, to power supply voltages out of range, etc.
Each of the above sensors turns an input value (number/string/etc) into a numeric value, which adds to the health score. That health score is what you see on the GUI’s sensors graph as show below. If the score reaches a danger threshold set by you (the administrator), HAAst can automatically run scripts or take some other action. And if that score reaches the fail over threshold, then HAAst will transfer control to the peer.
You can define as many or as few sensors as your like, and weight their scores as you like, to create a health scoring system meaningful to your environment. Similarly, you can set the critical and fail over thresholds to suit your needs.
Telium would be pleased to assist you in designing and setting up sensors. As well, we would be happy to create custom sensors to gather input from any device/service you wish (as a professional services project). We have interfaced with serial devices, thermostats, 1-wire networks, Bluetooth devices, and more.
Please remember that external sensors are completely optional and need to be tailored to your unique environment. Many clients run without any additional (external) sensors.
-
This reply was modified 5 years, 7 months ago by
WebMaster.
in reply to: Web GUI forbidden / unauthorized error on access #6744You most likely moved or deleted the password file associated with your GUI. Or you might have applied a HAAst update which (until the version released March 2018) would mistakenly remove that file during the automated update (install_updates.sh)
You can recreate the .htpasswd file as described in section 3.4.5 of the installation guide (Secure the Web Interface). I’ll repeat the Linux command here to save you from digging out the manual:
htpasswd -c /usr/local/haast/web_interface/.htpasswd user1
The above line will create username ‘user1’ and then prompt for a password to be associated with the username. To reset the password just repeat the process. Note that the “-c” parameter creates a new file, so it removes other username/password combinations from this directory. If you want multiple usernames for this directory please check the man page of htpasswd for help.(After March 2018 if you run install_updates.sh to update your version, then your .htpasswd file will be preserved)
in reply to: License file rejected #6743HAAst uses “activation”, exactly as Microsoft Windows does. If you change your network card in Windows then Windows will operate in non-activated mode. You have to reactivate your Windows installation with Microsoft by phone or online in order to reactivate your license. Like Windows, HAAst detects the change and switches back to the Free Edition. Once you reactivate it returns to the commercial edition.
We face the same problem as Microsoft or any other software vendor: piracy. So unfortunately activation is necessary. But, we are looking at an online licensing system so that you can move your license around (so long as your PBX has an internet connection). Many emergency call centers will not allow a dependence on the internet to permit their systems to operate, so for them we will always offer an activation license file.
Please ensure you only activate HAAst once your hardware platform is finalized (this includes virtualized hardware). Most customers won’t need to reactivate an installation unless something fails (which might happen every few years, and we’re happy to reactive for you). If you need to reactivate every few months then you are doing something wrong!
in reply to: Permanent ban of IP #6741Yes. First, ensure that the ‘managemanual’ setting in the [banip] stanza of your secast.conf file is set to true.
Next, from the telnet interface to SecAst manually add the IP address to the ban list. For example:
SecAst>banip add 1.2.3.4
Issued request to add IP 1.2.3.4. Check event log for errors, or use ‘banip list’ to confirm add
SecAst>After that SecAst will leave the address in your blocked IP list, and never expire the ban.
in reply to: Max call capacity #6740The unlimited edition of HAAst does not impose any limit on number of simultaneous calls. From a practical standpoint HAAst is limited only by the capacity of your hardware, and the design of Asterisk.
We have HAast “PBX” deployments with over 14500 phone sets, 3500+ simultaneous calls, 800 call setups per minute, etc. HAAst operates perfectly in those environments. We also have HAast “gateway” deployments (i.e. HAast is bridging calls and offering services, but is part of a larger telephony service) with tens of thousands of simultaneous calls.
Properly sizing your host to support a desired call volume is outside the scope/role of HAast. But you need to consider many factors like transcoding (CPU heavy), recording (I/O heavy), bridging/conferencing, etc. As well, placing Asterisk inside a container or virtual machine imposes other limits on CPU and IO availability. If Telium is provide a turnkey HA solution we assume responsibility for scaling the solution to meet your needs; however, in general Telium does not providing hardware scaling assistance.
For companies requiring HA for call volumes beyond what a single host/VM/container can support, we recommend running multiple clusters to achieve HA. To keep the clusters in sync we recommend adding our PBXsync product.
If you are at the point of sizing hardware for purchase, we recommend adding 5% to CPU capacity, and 500MB to memory capacity for HAast, above what is demanded by Asterisk. (Which in the overall scale of these systems is negligible).
in reply to: QObject error message starting with -f parameter #6739I’m glad you are running the program with the -f parameter first, since that is a great way to identify problems.
When you see this type of error (with “QObject”) it usually means something is seriously wrong with the system, often relating to missing prerequisites, libraries, etc. Our program is crashing as a result of these problems, and is mostly likely shutting itself down.
Please review the installation instructions carefully, in particular the prerequisite packages needed, and you will find the problem.
in reply to: Creating a HA LCR / Proxy server (cluster) #6738As an additional note, you may wish to have a look at our LoDi product. It can act as an outbound proxy / LCR / and more. LoDi includes an HA option so it might be an easy way to achieve what you are looking for.
in reply to: Creating a HA LCR / Proxy server (cluster) #6737Yes HAAst can create a High Availability LCR / Proxy server cluster for you.
HAAst’ health detection will automatically shutdown a failing LCR/proxy and start a healthy LCR/proxy. HAAst’s synchronization capabilities will keep the dialplan in sync between peers, and also all of your cost data in the MySQL tables. HAAst can even start/stop unrelated services (perhaps relating to LCR), customize data unique to each LCR/proxy, and more.
HAAst has no problem with 1500 simultaneous calls, or setups, per second.
in reply to: Hacker score causing blocked user #6736That message means that the source IP is known to be used by hackers and that the connection attempts are most likely hack attempts. However, the user is using valid credentials!
So if this blocking is correct then change the account name and secret for the account in question.
If you don’t want SecAst to block this IP address you can raise the threshold for hacker detection.
-
AuthorPosts