CouchDB beam uses 100% of CPU and network


While playing around with CouchDB I have observed that my system was running really slow (actually first of all I notice it heating) and the System Monitor showed that both CPU was at 100% and network usage was very high. The responsible for this was beam.

Doing some Google research I found that beam was related to CouchDB but the only thing that I was trying to do is connecting to a CouchDb server and after getting a DB Connector using a wrong user/password combination, try to create a document.

Debugging my Java code I found that the system was running some code in Apache HTTP client that for some reason did not finish (showing an error or throwing an Exception) but keep trying and trying.

Trying to find the problem I downloaded the sources of the latest version (4.2) of httpclient, httpclient-cache and httpcore for debugging them and … surprise! the problem disappear. Then I downloaded the sources for version 4.1.1 (the one that I was using before) and failed again.

Conclusion: If you have problems with beam and you are using apache-http version try upgrading it.

NOTE:

Advertisements

Installing CouchDB 1.2 in Ubuntu 12.04


Sad enough, seems that there is no version of CouchDB 1.2 in Ubuntu Software Sources -you will find 1.0.1 but it is pretty old- so you have to download and compile it!

Installing Ubuntu Desktop 12.04 LTS

The very first thing is going to Ubuntu site and download the ISO image (I have chosen Desktop -easier administration- and 64 bits edition).

Since I run it in a virtual machine (with VmWare Fussion) and after a couple of unsuccessful (easy) installations I’ve decide to go with manual installation.

My (non-default) Settings are:

  • Processor & RAM: 1 CPU, 3072 MB.
  • Hard Disks: SCSI Disk, 40 GB (Split in 2 GB files).
  • Network: NAT (Generated MAC address – click on Advanced Options and then Generate).

Doing it manually went pefectly well other than the resolution originally set during the installation (5120 x 3200) that I switched to 1024×768 by clicking on an icon on the top right corner -a gear- and then choosing System Setting… > Displays).

During the installation I have chosen NOT to download upgrades / updates so I did it just after the reboot. The easiest way is open a terminal and run:

$ sudo apt-get update
$ sudo apt-get dist-upgrade
$ sudo reboot

It downloads from your source repository the description of the updates and then the second command upgrade packages installed and outdated.

NOTE: I reboot the system since the kernel was updated too.

Installing CouchDB 1.2

  • Start installing some packages that are going to be needed:
$ sudo apt-get install g++
$ sudo apt-get install erlang-base erlang-dev erlang-eunit erlang-nox
$ sudo apt-get install libmozjs185-dev libicu-dev libcurl4-gnutls-dev libtool
  • Then go to CouchDB site and choose to download sources.
  • In a terminal, go to the folder where you have downloaded the file and run:
$ ./configure
$ make
$ sudo make install
  • Start couchdb
$ sudo couchdb
Apache CouchDB 1.2.0 (LogLevel=info) is starting.
Apache CouchDB has started. Time to relax.
[info] [<0.32.0>] Apache CouchDB has started on http://127.0.0.1:5984/
  • Check that is actually running using curl (install it if needed):
$ sudo apt-get install curl
$ curl -X GET http://localhost:5984
[info] [<0.361.0>] 127.0.0.1 - - GET / 200
{"couchdb":"Welcome","version":"1.2.0"} $

And, hopefully, that’s it!

Setting CouchDB as service

Maybe you don’t want to start CouchDB manually each time you reboot your machine.

In that case some few things to do:

  • Add couchdb user and group.
$ sudo adduser --disabled-login --disabled-password --no-create-home couchdb
Adding user `couchdb' ...
Adding new group `couchdb' (1001) ...
Adding new user `couchdb' (1001) with group `couchdb' ...
Not creating home directory `/home/couchdb'.
Changing the user information for couchdb
Enter the new value, or press ENTER for the default
 Full Name []: CouchDB Admin
 Room Number []:
 Work Phone []:
 Home Phone []:
 Other []:
Is the information correct? [Y/n] Y
$
  • Check/set right owner for files and folders (alternatively, you might have it installed in /var instead of /usr/local/var.
$ sudo chown -R couchdb:couchdb /usr/local/var/log/couchdb
$ sudo chown -R couchdb:couchdb /usr/local/var/lib/couchdb
$ sudo chown -R couchdb:couchdb /usr/local/var/run/couchdb
$ 
  • Link couchdb service script to /etc/init.d.
$ sudo ln -s /usr/local/etc/init.d/couchdb  /etc/init.d
$ 
  • Configure service to start when you enter / change levels:
$ sudo update-rc.d couchdb defaults
 Adding system startup for /etc/init.d/couchdb ...
   /etc/rc0.d/K20couchdb -> ../init.d/couchdb
   /etc/rc1.d/K20couchdb -> ../init.d/couchdb
   /etc/rc6.d/K20couchdb -> ../init.d/couchdb
   /etc/rc2.d/S20couchdb -> ../init.d/couchdb
   /etc/rc3.d/S20couchdb -> ../init.d/couchdb
   /etc/rc4.d/S20couchdb -> ../init.d/couchdb
   /etc/rc5.d/S20couchdb -> ../init.d/couchdb
$ 

Next time that you reboot your machine you should be able to access CouchDB with no additional actions.

Possible errors / problems

This is a list of errors that I’ve seen not following these steps:

  • When erlang is not installed you get:
checking for erl... no
configure: error: Could not find the `erl' executable. Is Erlang installed?
  • Original documentation says to install libmozjs-dev but it is not in Ubuntu Software Sources: Install libmozjs185-dev.
  • When g++ is not installed, configure phase fails complaining about SpiderMonkey not installed!!! Please, install g++ and try again. This is the error that I got:
checking for JS185... yes
checking for JS185... yes
checking for JS185... yes
checking for jsapi.h... no
checking js/jsapi usability... no
checking js/jsapi.h presence... no
checking for js/jsapi.h... no
configure: error: Could not find the jsapi header

Are the Mozilla SpiderMonkey headers installed?
$ ls /usr/include/js/jsapi.h
-rw-r--r-- 1 root root 136254 Mar 31  2011 /usr/include/js/jsapi.h
$
  • When erlang-eunit is not installed you get errors at compile time (even that configure finish correctly). Install erlang-eunit to fix it! This is the error that I got:
$ make
make  all-recursive
make[1]: Entering directory `/home/onabai/Downloads/apache-couchdb-1.2.0'
Making all in bin
make[2]: Entering directory `/home/onabai/Downloads/apache-couchdb-1.2.0/bin'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/home/onabai/Downloads/apache-couchdb-1.2.0/bin'
...
Making all in mochiweb
make[3]: Entering directory `/home/onabai/Downloads/apache-couchdb-1.2.0/src/mochiweb'
/usr/bin/erlc  mochifmt.erl
./mochifmt.erl:none: error in parse transform 'eunit_autoexport': {undef,
                                              [{eunit_autoexport,
                                                parse_transform,
                                                [[{attribute,1,file,
                                                   {"./mochifmt.erl",1}},
                                                  {attribute,7,module,
                                                   mochifmt},
                                                  {attribute,8,author,
                                                   'bob@mochimedia.com'},
                                                  {attribute,9,export,
                                                   [{format,2},
                                                    {format_field,2},
                                                    {convert_field,2},
                                                    {get_value,2},
                                                    {get_field,2}]},
                                                  {attribute,10,export,
                                                   [{tokenize,1},
                                                    {format,3},
                                                    {get_field,3},
                                                    {format_field,3}]},
                                                  {attribute,11,export,
                                                   [{bformat,2},{bformat,3}]},
...
                                               {compile,
                                                '-foldl_transform/2-anonymous-2-',
                                                2},
                                               {compile,foldl_transform,2},
                                               {compile,
                                                '-internal_comp/4-anonymous-1-',
                                                2},
                                               {compile,fold_comp,3},
                                               {compile,internal_comp,4},
                                               {compile,internal,3}]}
make[3]: *** [mochifmt.beam] Error 1
make[3]: Leaving directory `/home/onabai/Downloads/apache-couchdb-1.2.0/src/mochiweb'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/onabai/Downloads/apache-couchdb-1.2.0/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/onabai/Downloads/apache-couchdb-1.2.0'
make: *** [all] Error 2
$
  • If you did not install erlang-nox package you get an error when try to run CouchDB. Install it and re-run CouchDB. The error is something like this:
$ sudo couchdb
{"init terminating in do_boot",{{badmatch,{error,{"no such file or directory","inets.app"}}},[{couch,start,0},{init,start_it,1},{init,start_em,1}]}}

Crash dump was written to: erl_crash.dump
init terminating in do_boot ()
$

Linux setup


As I said in the previous post (Setup), I have chosen a desktop distribution in order to save some time editing files but some configuration is still needed.

Set Static IP address

It’s nice but also bad that each time that you restart a computer you get a different IP address, specially if this is a server (or provide some sort of service) and then you need to configure the clients.

Since our Linux box is going to be in a separate machine, lets set a static IP address in order to easily find it.

  1. On you Mac go to /Library/Application Support/VMware Fusion/vmnet8and edit dhcp.conf adding at the end of the file the following lines:
    host couchdb64 {
        hardware ethernet 00:50:56:28:99:48
        fixed-address 172.16.221.21;
    }

    You get hardware ethernet from the settings of the Virtual Machine > Network > Advanced Options and then click on Generate.

  2. Go to System Setting > Network > Wired > Options >  IPv4 Settings and Edit Addresses
    Address: 172.16.221.21,
    Netmask: 255.255.255.0 
    Gateway: 172.16.221.2

Install CouchDB

We are going to do this from Ubuntu repositories (even that you might opt to go with a version downloaded from Apache CouchDB site but then you have to compile it by yourself following their instructions). This is pretty simple:

couchdb:~$ sudo apt-get install couchdb
Reading package lists... Done
Building dependency tree 
Reading state information... Done
The following extra packages will be installed:
 couchdb-bin erlang-base erlang-crypto erlang-inets erlang-mnesia
 erlang-public-key erlang-runtime-tools erlang-ssl erlang-syntax-tools
...
* Starting database server couchdb [ OK ] 
Setting up erlang-syntax-tools (1:14.b.4-dfsg-1ubuntu1) ...
Setting up libsctp1 (1.0.11+dfsg-2) ...
Setting up lksctp-tools (1.0.11+dfsg-2) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
couchdb:~$

Change CouchDB listening IP address/port

Now, the problem is that CouchDB is only listening at localhost (127.0.0.1) and we want to access from other machines so we need to change CouchDB IP/port.

Edit /etc/couchdb/local.ini file and look for the label [httpd] and there is bind_address  property that you should edit and change to 172.16.221.21 (your static IP address).

[httpd]
;port = 5984
bind_address = 172.16.221.21

You might also be willing to set bind_address to 0.0.0.0 for making CouchDB list to all network interfaces.

Restart CouchDB server:

couchdb:/etc/couchdb$ sudo service couchdb restart 
 * Restarting database server couchdb [ OK ] 
couchdb:/etc/couchdb$

Open your web browser in the Mac and try to access http://172.16.221.21:5984 and you should see {“couchdb”:”Welcome”,”version”:”1.0.1″} as response. If you open the URL http://172.16.221.21:5984/_utils/ then you get CouchDB futon.

Setup


I´m going to be using for this project an Apple MacBook Pro but actually most of the software used is available also for Linux and Windows platforms. So hopefully you are not going to have problems replicating the setup.

List of elements of the setup:

  • Application server: Apache Tomcat 7.0 on the MacBook Pro. Apache Tomcat is downloaded as binary from Apache site and just uncompressed in my project folder.
  • CouchDB Server: on an Ubuntu Desktop 12.04 LTS (64 bits) virtual machine running in the MacBook Pro using VmWare Fusion (I love VmWare products since many years and likely that VirtualBox, Parallels,… are as good but I’ve started with VmWare and they run well for what I need and typically I can afford their prices).
    Yes! One question about the Linux distribution that I’ve chosen… since this project is likely to keep me bussy for some time, I got the latest. And about “Desktop” and not “Server” because I don’t want to have to do some manual configuration as switching keyboard layout, configure IP address… and whenever I want to do some (serious) performance testing I will try to find another machine and then I will install server. Nevertheless, I do a lot of manual (console) operations specially for those that I use to do quite ofter (start services, install packages,…).
  • CouchDB is directly installed from Ubuntu repositories (sudo apt-get install couchdb).
  • Ubuntu machine uses NAT and fixed IP address just to make sure that each time I reboot it I can easily find it.
  • IDE: IntelliJ IDEA (sorry! I’m not an Eclipse follower just because I’ve started using Idea years ago and I’m not about to switch as far as I don’t have to and Idea offers me what I need).

This is basically about it… Hopefully pretty simple.