May 2010

Tigase Server monitoring next step - remote monitoring via JMX/RMI, SNMP, HTTP

A few weeks ago I have written the article about active service monitoring in the Tigase server via XMPP. This mechanism allows for sending alerts to a selected set of JIDs. It allows also to pull statistics from the server on demand. It is very convenient way because you can track the server activity using any XMPP client. You can also receive warnings while you are online.

I received a good feedback about this feature but also some people pointed me out that it would be extremely useful to do remote monitoring of the service via different protocols as well. This would allow to integrate the Tigase server monitoring with external automated systems.

I have been working hard for a few last weeks to fulfill those requests and now all the basic server statistics are accessible via JMX framework as MBeans.  As MBeans they are automatically available to any JMX server. There are 3 servers available with the Tigase server: JMX/RMI default Java server. This protocol gives you the most flexibility and most control over the server. It allows not only to passively watch the server activity but also you can change the server parameters and interact with the system.

Two more JMX servers are available as well exposing data via SNMP and HTTP.

Less resources, higher throughput - 150k active connections

Thanks to Directi Company who offered their hardware for Tigase tests I had a unique opportunity to do more detailed load tests under the Java profiler control. Good environment teamed with a good tools allowed me to find places where some optimizations are possible and further improve the Tigase server. In my last tests, even though, the Tigase server was running under the Java profiler which consumes extra memory and CPU power I was able to achieve 152 584 concurrent, active connections. In this test the client side software was the main limitation. I had only 3 machines to generate client traffic. In theory it allows to create maximum 3 x 64k connections. In practice it is slightly less. On top of this the client generators use more resources than the Tigase server as they are significantly less optimized. As in previous test the hardware used to run Tigase server was: Pentium(R) D CPU 3.00GHz Dual Core, 4GB RAM, 144GB SATA HDD running on 64bit CentOS 5.1, Sun HotSpot(TM) 64-Bit Server VM (build 1.6.0_03-b05, mixed mode). Here are statistics taken at the end of the test:
Tigase Server version3.3.2-b889
Concurrent, authenticated connections152 584
Test total time 2h, 14min, 8sec
Memory concumptionabout 2.2GB
CPU usage (2 cores)below 80%
Total number of XMPP packets processed during the test40 407 254
Average packets per second processed10 077/sec
Average server response time466 millis
Database (running on localhost)MySQL

The rest of the test parameters and all conditions stay the same.

Tigase on Nokia N900

 I am at the XMPP Summit/FOSDEM now. Thanks to people form Nokia we have an opportunity to play and run some software on N900 devices and thanks to Florian from Flosoft we have SIM cards for data transfer.

What is the most tempting thing for the software developer? Well, at least for me is to run my software on any possible device.

Taking the opportunity to use N900 and play with it I just had to try to run Tigase on it. I have already tested Tigase on small computers, Netbooks with ATOMs and small amount of memory. Tigase passes all tests when run with 10MB of memory so I was pretty sure N900 has enough memory and power to run Tigase.

Tigase load tests on Nokia N900

Normally you would not call something a load test if there are just 40 user connections opened to the server.

This time, however the Tigase was running on the mobile phone which is itself an interesting achievement. One thing, however is to just run the application and another is to actually make it do something useful.

The Tigase running there was just the standard version you can download from the website, no single line of code was changed. And also the N900 was running all the standard applications as a fully functional mobile phone.

The only change I made to the phone was installation of the JRE and Bash shell, the rest remained unchanged and none of applications running on the phone was stopped.

All the connections to the server were established via mobile network and all the connections were encrypted with TLS.

XMPP service in own domain

Tigase.im is a public XMPP service available to everybody. We have just added a new feature - you can now have own domain and register XMPP accounts within the domain. If you want to experiment with XMPP server, setup own small service, you don't need now own server running somewhere. You can just use our public and free to all server. Please note, this service runs on the development version of the Tigase XMPP Server. While we try to make it as reliable as possible you may experience periodic restarts. On the other hand all the most recent stuff are available there. All the experimental code and new features. If you want to test your client, library or other code, this might be a good place to do it. All details on the front page.

Tigase server monitoring

This is our the most recent addition to the Tigase server implementation: automatic and self-monitoring tools inside the server. This stuff is exceptionally useful for advanced users and those who manage Tigase installations working under a high load.

There have always been statistics in the server accessible via ad-hoc commands which could provide you with lots of details about the server internal states, the load on the server, performance and so on. The disadvantage of using them was that you need to periodically check the statistics and actively monitor the server in order to detect that something might go wrong.

Alternatively you could write a bot which could pull statistics periodically, analyze them and send a notification if something goes wrong. This solved the problem with manual checking statistics but still monitoring of the server is somehow limited.

To overcome all the limitations and even expand possibilities to monitor the state of the Tigase installation we have added automatic monitoring tools to the server. They allow to early detect if something goes wrong or if the server is under high load and can't keep up with processing all the data.

It consists of 2 parts:

  1. Monitor Framework and API - which creates a basement for the monitoring system and allows interaction between a user and the monitoring system.
  2. Resource Monitors - which are small plugins responsible for monitoring certain resource. It's kind of Unix like approach to have small and simple applications responsible for just one thing.

Tigase Load Tests again - 500k user connections

I have had a great opportunity and pleasure to use Sun's environment both hardware and software to run load tests on the Tigase server for a couple of last weeks. Sun also ofered me something more. Their knowledge and help during the tests allowed me to improve Tigase and fine tune the system to get the best results. I would like to say great thank you to Deepak Dhanukodi, ISV Engineering from Sun who was a huge help.

The longest running XMPP service

I wonder what is the longest running XMPP service without a restart. I watch a few servers running Tigase but then I restart them from time to time for software update with new features or bug fixes. One of them is im.flosoft.biz which runs on quite old Celeron 1.33GHz machine with 200MB memory. Serves approximately 800 online users and works well for 75 days already without a restart. This is a development version of the Tigase 5.1.0 branch but indeed, Florian pushes me to release it as another stable version. I have found however an absolute winner in this 'competition'. The last time I checked it earlier today the Tigase uptime time was: 657 days, 6 hours, 53 mins, 21 secs. This is a service running on MS Windows machine for online gaming website and most of the user connections are web clients connecting via Bosh. It is still running and working well! I have also received a few other reports from people successfully running Tigase, as they say for 2 years without restart. If you have any specific data, number of days, the software version, please add a comment to this article. I would like to hear about your experience with long-running Tigase server.

XEP-0079 - Advanced message processing support in Tigase

I have just added an initial version of advanced message processing (XEP-0079) support in the Tigase server. I could not find any client side software offering this extension, so this was tested only with my own tools. Obviously this is not good enough so I made it available on the public service: Jabber.Me/Tigase.IM. This installation runs on the development version of the Tigase server. It offers all the news stuff I am working on, now including AMP. If you work on AMP too or have software supporting this already, please give it a try and send me your comments.

Get in touch

We provide software products, consulting and custom development services

Tigase, Inc.
100 Pine Street, Suite 1250
San Francisco, CA 94111, USA
Phone: (415) 315 9771

Follow us on:

Twitter

  • New Blog post: experiences with Tigase Messenger for iOS: https://t.co/0iP7bTfjCX 4 days 15 hours ago
  • Trying out our new iOS application and have questions? Check out documentation if you need help. https://t.co/omsDijTW1k 6 days 23 hours ago
Back to Top