Switching from MySQL5.5 to MariaDB5.5 (or maybe MariaDB 10.0?)

I’ve been using MySQL for at least 7 years now and it worked more or less fine.
But few days ago on http://webconf.lv conference author of both MySQL and MariaDB Monty Widenius persuaded me to try MariaDB.

Why switch?
Why not? It is fully compatible with MySQL (You will not loose any features and You don’t need to change anything in Your existing infrastructure) and it works faster and on top of it has additional nice features like virtual columns which were one of the things where NoSQL had advantage before.
Also SphinxSE is well integrated into MariaDB which is a plus for me because I am actively using full text search engine Sphinx in three projects.
You can read about improvements over MySQL here:

kb.askmonty.org/en/mariadb-versus-mysql-features/

The migration process
My personal production server uses Centos5.8 x64 with MySQL5.5 from remi repository. I uninstalled MySQL and installed MariaDB from their repository but I ran into libmysqlclient conflicts because of old php-mysql extension which is still used in few of my projects. It took me almost 20 minutes to find solution and later I updated MariaDB knowledgebase website which is meant to be better alternative to MySQL official documentation:

kb.askmonty.org/en/how-can-i-upgrade-from-mysql-to-mariadb/

And now my production server runs MariaDB and so far it runs without any problems. Unfortunately I didn’t have spare time to make my own “real world” benchmarks but it seems to use little bit less resources and strangely enough it works well without ANY customizations in configuration which now has it’s own configuration folder in /etc/my.conf.d with separate files which make it easier to configure it.
For example, with MySQL default settings my server would use constantly 50% of one CPU core and this forced me to invest quite a long time to fine-tune config and use such tools as mysqltuner.pl.
I am quite confident if I will fine-tune MariaDB I will get much better improvement in speed and resource usage even if I’m OK with current metrics.

Future of MariaDB
If you are still not so impressed with current version of MariaDB and thinking about sticking with Oracle’s MySQL You should at least check progress of MariaDB 10.0 which will be next major release.
First Alpha version was released yesterday and as Monty told audience of Webconf it will be already stable at the end of this year. One of more interesting additions of 10.0 is Cassandra storage engine:

https://kb.askmonty.org/en/what-is-mariadb-100/

MariaDB team has the best software engineers from MySQL and they are committed to keep MySQL/MariaDB open unlike Oracle which tries to kill it. I hope community will spread the word about MariaDB to all MySQL community so more people will switch to MariaDB which will help keep it open and community oriented.

You can get started on mariadb.org/

newrelic.com web performance testing tools and server monitoring

New Relic is web application performance management tool for Your dedicated server or VPS.

It supports PHP, Java, Ruby, Python and .NET web applications.

With New relic You can easily find out problems in a server because it sorts all problems by importance and shows what is more important.

New Relic has decent MySQL database monitoring features which can help You find tables which are used the most and which needs attention the most. It also shows slow queries. It also supports NoSQL but I don’t have any apps which use it so I don’t knwo how good is NoSQL support.

If You don’t have basic server resource monitoring yet on Your server You can also enable CPU, Disk, Memory and Network monitoring with nice historical data.

The best thing about New Relic is that both web app management and server monitoring are very easy to install. It took me 5 minutes from installing both services to start checking out fancy graphs about my server performance on their website

I am little lazy to explain all the cool stuff about this service so take 10 minutes of Your time to register:
http://newrelic.com

When creating Your free account You automatically get trial of PRO version for 15 days after which You can continue to use free version.

Here are some screenshots from my server:
http://imgur.com/a/eAuFU

xdebug profiler for PHP and kcachegrind to optimize site speed

To enable xdebug profiling You just need to enable xdebug extension (add zend_extension=”/usr/lib64/php/modules/xdebug.so” for example if You are on 64bit Centos Linux distribution) and edit Your PHP configuration by creating separate xdebug.ini file or add these lines directly in php.ini:

xdebug.profiler_enable = 1 # by default profiler is not enabled
xdebug.profiler_output_dir = “/var/log/xdebug” #this is the directory where xdebug profiler generated files will be located
xdebug.profiler_output_name = “callgrind.%R::%u” #to work with kcachegrind output files must have callgrind. prefix (by default xdebug uses cachegrind.out. prefix) change this so You don’t need to manually rename files to callgrind. prefix.

Please don’t enable xdebug profiler ir production environment because in few seconds it can generate GIGABYTES of data and it will most likely crash Your webserver.

Here is an example what we can get from kcachegrind (data is from one of my first php projects):

Click to enlarge:

kcachegrind sorts everything by importance so on top You can see PHP actions which use the most resources so You don’t need to guess where to optimize.

For example, I had ancient multi-language library in this project where I created 900 method calls for each of 900 translations. With one little fix now I make only 1 method call from multi-language library.

Unfortunately I couldn’t enable Profiler on XAMPP 1.7 and I tested this feature on production server for a few seconds.

There is more stuff about xdebug to write about so I am quite sure there will be Part II of this post

Opensource web based video converter wap4.org

My first open source project is Wap4 video converter.

Project started on December, 2010 as project for my University and first time I committed source code on Github in February, 2011 after 3 months of work (I didn’t understand flow of version/revision control systems in those days yet)

Some facts:

• It’s based on CodeIgniter framework.
• All files are converted using FFMPEG.
• Links to video files are extracted using youtube-dl Python video grabber

UPDATE (2012.03.05):
What is current status of the project?
I don’t have much time to improve wap4 but I am still adding video formats and updating source code components (for example, youtube-dl)

First thoughts about Nginx webserver (optimizing my webserver Part 2)

A little while ago I was writing about optimizing overall webserver performance for my Apache2 webserver jurismalinens.wordpress.com/2011/10/23/optimizing-apache2-web-server-and-php/ but I still wasn’t satisfied with performance.

So I decided to play with some lightweight web servers. I already have little experience with lighttpd which was used as busy video streaming website and it performed very well for this sort of job but I wanted to try something new and I choosed nginx. Installation on Centos 5 was an easy part and configuration was also an easy task (nginx documentation for adding php was little bit out of date). Because I have few IP addresses available I decided to run Apache2 with Nginx side-by-side.

Performance
I didn’t get significant reduction of CPU usage but I got improvements in memory usage and I got better response times. Nginx handled it’s tasks perfectly and it helped me to find bottlenecks more clearly- PHP and MySQL. Most of resources were eaten by those two beasts.

Conclusion
By installing lightweight webservers You will not get rid of all performance issues and it is only one of many things You need to consider to optimize resource usage of Your webserver.

I’ve created new website for mobile phones- www.nextmobi.net

After more than 6 months of work I  officially launched nextmobi.net and mobile version is available on m.nextmobi.net.

Programming

It’is created in PHP scripting language and I used Cibonfire which is based on Codeigniter as development platform. It’s fairly new but I have no regrets by choosing this platform.  Development time most certainly would be longer if I choose to use Codeigniter without cool features of Cibonfire and it’s documentation.  There are many things which aren’t finished yet but I felt that it’s now the time to show the world my new website.

Goals

I don’t have grand ambitions (maybe I should have?) and I don’t want to be direct competitor of GSMArena or one of the other big names in this industry but I hope to get small part of the cake by providing some unique features.

Programming is fairly small part of success and I hope to have rich content and large community so big part of it is marketing.

What’s next for http://www.nextMobi.net?

Of course more visitors are needed to make website more alive. I will work on social networks and will further investigate techniques how to engage visitors and how to keep them coming back to NextMobi. I’ve added Kissinsights for some parts of website and I thinking of using Hellobar. I will also be working on better integration of social networks. If I have more interesting ideas I will post them here

So don’t be shy and  visit www.nextmobi.net and post what do You think about it

Optimizing Apache2 web server and PHP

A week ago I started to think about optimizing performance of my web server because there was quite high load on my server- about constant 30% CPU usage.

The biggest problem was with MySQL which had about 700 queries per second (mostly from my old mobile statistics portal: statistics.infowap.info). So I removed queries which were not essential, optimized some indexes, combined some queries by joining them and I got about 40% less CPU usage for MySQL and about 400 queries per second. Of course big thanks go to these two fantastic resources for optimizing SQL queries:

http://use-the-index-luke.com/ found this resource thanks to Reddit

http://www.mysqlperformanceblog.com/ but this while Googling about MySQL…

I am not MySQL guru so I didn’t try to optimize my SQL server even further but I still got quite high CPU load. And now I started to think how I can optimize Apache2 web server.

Some of the things I done:

AllowOverride All changed to AllowOverride None and htaccess Rewrites moved from .htaccess to Apache configuration files

Disabled unnecessary Apache modules, for example, mod_status, mod_info, mod_speling, mod_proxy_balancer, mod_proxy_ftp, mod_proxy_connect, mod_dav_svn, mod_wsgi

KeepAlive On, KeepAliveTimeout 2 gave me the best performance.

But still I was not satisfied so I got my hands onto PHP optimization. Different virtual hosts needed different PHP modules but every module was enabled for every virtual host (I have different php.ini for every host). I disabled most of php modules globally from /etc/php.d/ and enabled php extensions per virtualhost basis and results were impressive!  I got about the same performance increase as by optimizing indexes and MySQL configuration.

In summary I got huge performance increase. Before I had about 30% load on all 4 CPU cores but now I have 10-15% CPU usage for four CPU cores. basically I could run all my websites on much “weaker” hardware and maybe even I can move back to VPS solutions now instead of using dedicated server with all problems associated with it