To support Master -> Slave/Master -> Slave configuration, one needs to add log-slave-updates in my.cnf for the node acting as a slave as well as master for other slaves. If this is not done, no data from the master will get replicated to the final slave and the replication will work only between master and 1st level slave.
The log-slave-updates option enables replication instructions from Master 1 to be written to Master 2’s binary log so that they can then be replicated to the true slaves.

Advertisements

ProgrammableWeb now has a list of 1000 Web APIs. They crossed this milestone last week. They had started with 32 in summer of 2005.

Checkout the APIs neatly arranged/categorized at http://www.programmableweb.com/apis/directory

Here are some key stats from ProgrammableWeb, about 63% of the APIs use REST, 22% SOAP and remaining XML-RPC, Atom, JavaScript

Number of API providers in top 15 categories from ProgrammableWeb

Number of API providers in top 15 categories from ProgrammableWeb

Protocol usage by APIs from ProgrammableWeb

Protocol usage by APIs from ProgrammableWeb


Code reviews and walkthrough help a lot in maintaining a tab on coding practice and allocation/freeing of memory in a application. But over a period of time an application grows with additional number of features being added over a period of time. So it’s always better to have an automated solution to find any memory related issue with your application.

Valgrind is a great tool that helps you do that. For information about valgrind please refer to following links:

Homepage: http://valgrind.org/
Downloads: http://valgrind.org/downloads/
Valgrind quick start guide: http://valgrind.org/docs/manual/QuickStart.html
Complete user manual: http://valgrind.org/docs/manual/manual.html

A simple example of using Valgrind to detect memory issues with an Apache+FastCGI web app written in C on a Linux box.

Say that the FastCGI based web app is already in production and running on your servers and you want to use valgrind to check memory related issue, follow following steps.

First install valgrind on the machine where you want to run the web app with valgrind do following:

# wget –c “http://valgrind.org/downloads/valgrind-3.3.1.tar.bz2”

Replace http://valgrind.org/downloads/valgrind-3.3.1.tar.bz2 with the latest available version (as bugs found with previous releases would have been fixed)

# bunzip2 valgrind-3.3.1.tar.bz2
# tar –xvf valgrind-3.3.1.tar
# cd valgrind-3.3.1
# ./configure
# make
# make install

If all the steps go thru properly valgrind will be built and installed in /usr/local/bin. Make sure that it’s in the PATH variable.

Now say the FastCGI configuration looks like in the httpd.conf file:

<IfModule mod_fastcgi.c>
FastCgiIpcDir /tmp/fcgi_ipc/
FastCgiServer htdocs/MyApp
</IfModule>

Instead of changing the whole configuration just write a simple shell script in a directory other than the one containing the actual MyApp FastCGI application

# cat > MyApp
#!/bin/bash
/usr/local/bin/valgrind -v –tool=memcheck –leak-check=full –log-file=/tmp/MyAppValgrind.log /path/to/my/app/ActualMyApp

Then press CTRL+D

Now stop Apache, move the MyApp (actual FastCGI application binary) to ActualMyApp.

Replace the existing MyApp with our new script, make sure it has execute permissions.

Start Apache, you are done. Valgrind will log the issue in /tmp/MyAppValgrind.log file

Note: Compile your program with -g to include debugging information so that Memcheck’s error messages include exact line numbers.

Information on Valgrind Memcheck error messages is available @ http://valgrind.org/docs/manual/mc-manual.html#mc-manual.errormsgs

Happy debugging 🙂


Komli today launched the “algoGod contest” for machine learning, math, genetics, and algorithm experts.

http://www.komli.com/algogod/

Contest winner to receive Rs. 2,00,000

Start date: October 15th 2007.
Entries must be submitted on or before November 14th 2007.
Results will be declared on December 31st 2007.

Have you ever wondered if you are the best algorithms expert on the planet? Have you ever thought, “I know I can beat everyone, just let me prove it?” Well Komli’s algoGod contest is for you, it’s your chance to show the world how smart you really are!

The algoGod contest seeks to crown one expert as the ‘Algorithms God’. How are we going to do this? Well, the proof is in the pudding! We want every contestant to solve a common problem, and whoever is best will receive the algoGod prize!

A little more background:

Komli lives in the world of online advertising, and online advertising is rife with opportunity with complex algorithms based on cutting edge topics such as machine learning, data mining, graph theory, etc. Online advertising is growing at a very fast pace, and the number of variables affecting the performance of an online ad has been growing at an even faster pace. Komli is devising methods for maximizing the yield of online advertising using advanced statistical machine learning methods over large-scale systems. This is a very interesting and complex algorithm problem.

Komli is currently using a set of algorithms for maximizing the yield of online ads, collectively called ‘Yin-Yang’. There are a lot of interesting alternative approaches to Yin-Yang that have yet to be tried. Komli is interested in determining if any of these alternative approaches can beat Yin-Yang by making better predictions.

Komli will provide participants with anonymous ad impression data and a prediction accuracy bar that is 50% better than what Yin-Yang can do on the same training data set. Participants’ solutions will be judged by ‘Time complexity’ and ‘Space complexity’ criteria. The participant whose solution works best will receive Rs. 2,00,000, bragging rights and an opportunity to work with Komli. Of course, participants have to share their method and code with Komli. Eager participants can signup for the contest by filling the form on the left. Also, please let us know of any questions at algogod@komli.com.


Simple Tooltips

30Sep07

http://codeeazy.com/mktooltip/

I wanted a tooltip implementation for my web page, and was looking around. I looked a number of open-source options (see 40 tooltip scripts here and 20 here) and one commercial library. I liked the functionality however each of them was 7, 8, 10 or 25 KB in size, which I thought was too much to implement just a simple tooltip. So, I thought how about I write a tooltip library myself.

Here is an implementation, check it out at http://codeeazy.com/mktooltip/. I implemented this in 818 bytes of code, and about an hour of coding. So one would think, there must be something wrong. Well, it works. And is cool!



Amazon EC2

19Sep07

The Amazon Elastic Compute Cloud (Amazon EC2) web service provides you with the ability to execute your applications in Amazon’s computing environment.

Although there is no provision for SLA which might be required by big wings, it’s a good service model for computing. Use what you want and pay for it, not more not less. No email and applications to get a server, you want one now, go and provision it and start using it, it’s that simple. Small and big companies can both use this service and reduce a lot of in-house maintenance cost, not in some cases where in you want more control ;).

For using Amazon EC2 one will need to do the following:
1. Create an Amazon Machine Image (AMI) containing all your software, including our operating system and associated configuration settings, applications, libraries, etc. Think of this as zipping up the contents of your hard drive. Amazon provides all the necessary tools to create and package the AMI.
2. Upload this AMI to the Amazon S3 (Amazon Simple Storage Service) service. This gives Amazon reliable, secure access to our AMI.
3. Register our AMI with Amazon EC2. This allows Amazon to verify that your AMI has been uploaded correctly and to allocate a unique identifier for it.
4. Use this AMI ID and the Amazon EC2 web service APIs to run, monitor, and terminate as many instances of this AMI as required. Currently, Amazon provides command line tools and Java libraries, and we can directly access our SOAP or Query based APIs.
One can also skip the first three steps and choose to launch an AMI that is provided by Amazon or shared by another user.

Once the above is done one have an AMI ID that the user can use to start server instances. For better management user will need to write various utilities for keeping information on the DNS for our server instances, so that user can connect to them. To check the status, say that the instance goes down due to some reason and comes back up the DNS for the server instance will change, user need to know the new one to access the server instance.If the DNS changes then all the utilities transferring data to the server instance have to be notified to start sending the data to the new address. Also as the data is not persistence on the server across reboots, if it restarts user will need to develop a mechanism to some how restore data on the server for processing. Amazon again provides a solution by providing Amazon S3 where one can store data persistently and accessing it from anywhere.

Here are some links for more information:

Getting started

Amazon EC2


Anand and I have been talking about this blog for a while, there was just no time to write about some of our inner feelings about – coding. We have been working on, what many people would call “the most bleeding edge” technology. To give a little bit of background, we are the founding engineers at Komli. Komli is an early stage startup. Since last November we have been learning some very cool techniques of solving problems. When it comes to building a product, a typical engineering mindset, lots of times gets stuck in technical jargons, technical-coolness and sometimes religious-warfare between different technical solutions, languages etc. While, the most important thing when you are building a product is – to solve a problem. That’s it. You have to solve a problem, apply any technology, apply any language, apply any tool, use two languages or 3 languages – you gotta solve the problem. While, I obviously, am against making code unreadable or un-maintainable, and therefore mixing technologies or languages may be a no-no in some cases, however engineers mostly overdo that, by applying that rule too strictly, therefore loosing focus on – solving the problem; you gotta solve the problem, that’s the most important thing.

We have been solving numerous problems, in a number of areas, and sometimes in very cool ways.

We would use this blog to tell you about some of the cool ways of solving complex problems in simple ways. Some of what we have learnt. Some hard-learnt-lessons. I hope that you will like it.