How to install Glnemo2 in Ubuntu?

Introduction

Glnemo2 [1]  is an interactive visualization 3D program for nbody snapshots. With this software is possible to create videos and take snapshots of our nbody simulations. Glnemo2 also accepts Gadget2 [2] snapshots.

Installation

GLnemo2 requires qt libraries:

$ sudo apt-get install libqt4-dev libqt4-core libqt4-gui 

Once installed qt we can install Glnemo2 software in our system.

$ wget http://projets.oamp.fr/attachments/download/671/glnemo2_1.52.tar.gz
$ tar xvzf  glnemo2_1.52.tar.gz
$ cd glnemo2
$ qmake -revursive
$ make

Example

In the bin directory you can find the binary. One option is to update the PATH variable or simply:

$ ~/glnemo2/bin/unix/debug/glnemo2 snapshot_000 

This command will open the glnemo2 program with the snapshots_000. With the software we can rotate the image, change colours, view portions of the image (only represents the density or the masses) and create videos.

References

[1] http://projets.oamp.fr/projects/glnemo2
[2] http://www.mpa-garching.mpg.de/gadget/

NBody Simulations with Gadget2

Introduction

GADGET [1] is a freely available code for cosmological N-body/SPH simulations on massively parallel computers with distributed memory. GADGET uses an explicit communication model that is implemented with the standardized MPI communication interface.

Installation on Linux

GadGet2 requires of GSL (GNU Scientific Library [3]),  MPI (MPICH or OpenMPI) and FFTW. The HDF5 is optional.  In this post I will explain how to install GSL FFTW and Gadget2. In the most popular Linux distributions are available MPICH and OpenMPI package. 

GNU Scientific Library

# wget ftp://ftp.gnu.org/gnu/gsl/gsl-1.9.tar.gz
# tar xvzf gsl-1.9.tar.gz
# cd gsl-1.9
# ./configure --prefix=/opt/envhpc/gsl/1.9
# make 
# make install

Instalación de FFTW

# wget www.fftw.org/fftw-2.1.5.tar.gz
# tar xvzf fftw-2.1.5.tar.gz
# cd fftw-2.1.5
# ./configure --prefix=/opt/envhpc/fftw/2.1.5/ --enable-mpi --enable-type-prefix --enable-float
# make 
# make install

Gadget2 

# wget http://www.mpa-garching.mpg.de/gadget/gadget-2.0.7.tar.g
# tar xvzf gadget-2.0.7.tar.gz

As HDF5 is not installed, we need to comment this line in the Gadget2 Makefile.

# make 

Know issues: 

main.c:3:24: error fatal: drfftw_mpi.h: Do not exist the file or the directory.

It is needed to change the include drfftw_mpi.h by srfftw_mpi.h [2].

Example of simulation

The Gadget2 source code contains a set of examples. One of them is galaxy colision which runs two disk galaxies into each other, leading to a merger between the galaxies. Each galaxy consists of a stellar disk, and a massive and extended dark matter halo. This example uses plain Newtonian physics, with 20000 disk and 40000 halo particles in total.

Copy the galaxy.param file from parameterfile directory. It is needed to edit it and change the variables InitCondFile, OutputDir and OutputListFilename with the complete path. 

InitCondFile       /home/user/Gadget-2.0.7/ICs/galaxy_littleendian.dat
OutputDir          /home/user/Gadget-2.0.7/Gadget2/galaxy/
EnergyFile         energy.txt
InfoFile           info.txt
TimingsFile        timings.txt
CpuFile            cpu.txt
RestartFile        restart
SnapshotFileBase   snapshot
OutputListFilename /home/user/Gadget-2.0.7/Gadget2/parameterfiles/output_list.txt

And finally, launch the simulation with mpirun (It is needed MPI implementation - OpenMPI or Mpich2): 
 

$  mpirun -np 16 ./Gadget2 galaxy/galaxy.param 

How to represent the data?

After simulation, you probably think in represent the data. Splash software [3] allows to represent from snapshots generated by Gadget. The installation on Ubuntu requires of libcairo package:

# apt-get install libcairo2-dev

Download the source splash source code and edit build/Makefile from Splash Software and change:

X11LIBS= -L/usr/X11R6/lib -lX11

for this line:

X11LIBS=-L/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/libX11.so

Finally:

# make SYSTEM=gfortran
# make install

Once compiled, the program for gadget2 is gsplash, With this we can generate a set of png images and create, for instance, a video with ffmpeg.

$ gsplash snapshot_00*
    _                                                 _  
   (_)   _               _           _         _     (_)_
      _ (_)    ___ _ __ | | __ _ ___| |__     (_)   _  (_)
   _ (_)  _   / __| '_ \| |/ _` / __| '_ \       _ (_)    
  (_)  _ (_)  \__ \ |_) | | (_| \__ \ | | |  _  (_) _    
      (_)  _  |___/ .__/|_|\__,_|___/_| |_| (_)  _ (_)   
          (_)  (_)|_| (_) (_)  (_)(_) (_)(_) (_)(_)     

  ( B | y ) ( D | a | n | i | e | l ) ( P | r | i | c | e )

  ( v2.2.0 [16th Nov 2012] Copyright (C) 2005-2012 )

 * SPLASH comes with ABSOLUTELY NO WARRANTY.
   This is free software; and you are welcome to redistribute it 
   under certain conditions (see LICENSE file for details). *

 Comments, bugs, suggestions and queries to: daniel.price@monash.edu
 Check for updates at: http://users.monash.edu.au/~dprice/splash 
 Please cite Price (2007), PASA, 24, 159-173 (arXiv:0709.0832) if you 
 use SPLASH for scientific work and if you plot something beautiful,
 why not send me a copy for the gallery? 

 splash.defaults: file not found: using program settings
           6  filenames read from command line
 Using default M4 cubic spline   (2h) kernel

----------------------- snapshot_000 -----------------------
 >> reading default Gadget format <<
 time            :   0.16666699833882792     
 z (redshift)    :     5.00 (set GSPLASH_USE_Z=yes to use in legend)
 Npart (by type) :            0      140005       39616       96877           0           0
 Mass  (by type) :    0.0000000000000000        1.1077876463313410        0.0000000000000000        0.0000000000000000        0.0000000000000000        0.0000000000000000     
 N_gas           :            0
 N_total         :       276498
 N data columns  :           10
> allocating memory: parts =     276498 steps =      1 cols =   10
 positions       276498
 velocities       276498
 particle masses       136493
 gas properties            0
******************************************************************

An example of this simulation (video):

References

[1] http://www.mpa-garching.mpg.de/gadget/
[2] www.fftw.org/fftw2_doc/fftw_6.html
[3] http://users.monash.edu.au/~dprice/splash/

OpenCL extension: enable printf in Kernels

The OpenCL standard contains some extensions [1] related with memory optimisations , double presicion numbers and Device Fission. But, in the OpenCL standard is not available any function to print inside a kernel function. However commercial implementation such as Intel OpenCL or AMD APP include this funcionality [2]. The name of this extension depends of the OpenCL implementation , but we can know it with a specific call to OpenCL API.

Example

The following example shows a simple device query with OpenCL. The last line is shown all extensions available with Intel OpenCL Device. This name is cl_intel_printf.

$ ./oclquery
... 
========================================
Device: 0
     VENDOR:   Intel(R) Corporation
     NAME     :   Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
     VERSION: OpenCL 1.2 (Build 56860)
     EXTENSIONS: cl_khr_fp64 cl_khr_icd cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_intel_printf cl_ext_device_fission cl_intel_exec_by_local_thread
 ========================================

How to

All extensions in OpenCL follow the syntax:

# pragma OPENCL EXTENSION extension_name : behavior

where behavior can be enable or disable.

Extension for Intel OpenCL

// OpenCL kernel 
# pragma OPENCL EXTENSION cl_intel_printf :enable

Extension for AMD APP OpenCL

// OpenCL kernel 
# pragma OPENCL EXTENSION cl_amd_printf :enable

References

[1] OpenCL extensions: www.khronos.org/registry/cl/specs/opencl-1.2-extensions.pdf
[2] http://stackoverflow.com/questions/5782388/printf-function-doesnt-work-in-opencl-kernel

Patch for VMWare in Ubuntu 12

WMplayer [1] have a bug in Ubuntu Systems related with some VMware modules. The error is similar to the following message:

_______________________________________________________

Before you can run VMware, several modules must be compiled and loaded into the running kernel.

_______________________________________________________

When we compile the modules for VMware it is possible that the error persists. There is a patch that we can apply to our system [2]. We only have to download the patch and change the line with the specific version of VMplayer that we have installed.

$ mkdir patch_vmware

$ cd patch_vmware

$ wget 

$ tar xvzf vmware802fixlinux320.tar.gz

Then It is necessary to edit the  file and change the line 8 with the specific version of VMWare in the patch-modules_3.2.0.sh file.

  5 
  6 fpatch=vmware3.2.0.patch
  7 vmreqver=8.0.2
  8 plreqver=4.0.X

  9 

 10 ... 

Finally we run the script as administrator user.

$ sudo ./patch-modules_3.2.0.sh

References

[1] https://my.vmware.com/web/vmware/free#desktop_end_user_computing/vmware_player/4_0

[2]  http://askubuntu.com/questions/133285/vmware-player-4-0-3-on-ubuntu-12-04-32-bit

How to compile CUDA programs in Ubuntu 12.4

CUDA 4.1 compiler is now compatible with gcc 4.5. The last Ubuntu version (12.4) works with gcc 4.6. So we need a minor version to compile cuda programs and we need to install a previous gcc version:

$ sudo apt-get install gcc-4.5 g++-4.5

Then we need to compile with --compiler-bindir option and the version or gcc directory to work. The next line compiles an example with gcc 4.5. This gcc version has been installed previously in the system.

$ nvcc --compiler-bindir gcc-4.5 example.cu

References

[1] http://stackoverflow.com/questions/8598579/cuda-compiler-not-working-with-gcc-4-5

Update Wordpress

If you have a wordpress installed in your server it is possible that updates will be notified. In this case, we do not only need the appropriate permissions in the server, besides apache has to be the propierary of the wordpress files [1]

When we upgrade to new version from Menu, the typical error is:

Could not copy files. Installation Failed wordpress

If you work with Ubuntu or Debian, these files (wordpress files and directories) have to be of www-data user. So we need to change this in the server. 

$ sudo chown www-data.www-data blog/* -R

Then we can upgrade Wordpress from Menu (Tools -> Update)

References

[1] http://codex.wordpress.org/Updating_WordPres

C parallel modules for Python

It is easy to create C/C++ modules for Python using the API [1]. In this post I present how to write a very simple math module for Python with OpenMP. 

Example

The aim is to create a parallel math module for python. We import numpy to create arrays and vectors.

1 2 3 4 5 6 7 8

#! /usr/bin/python # File: test01.py import parallelm #our module import numpy as np vector = np.arange(1000000, dtype=np.float) result = parallelm.vadd(vector, vector)

The parallelm.vadd function uses OpenMP. This function implements the sum of two vectors.

Compile with setup.py

The easiest way to compile our module is with setup.py function from distutils. In this function we must specify the correct flags to use OpenMP.

1 2 3 4 5 6 7 8 9 10 11 12 13 14

from distutils.core import setup, Extension import numpy as np module1 = Extension('parallelm', sources = ['src/parallelmath.c', 'src/math_omp.c'], extra_compile_args=['-fopenmp', '-fpic'], extra_link_args=['-lgomp']) setup (name = 'PackageName', version = '1.0', description = 'This is a demo package', ext_modules = [module1], include_dirs=[np.get_include()], )

In line 3 we specify the C source code and extra compile arguments. In this case we compile with "-fopenmp -fpic" options. Now we can compile with build option.

$ python setup.py build

C source code

Now we must implement the source code, so we have to interact with Python variables. From my view point this is the hardest part

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

#include <Python.h> #include <numpy/arrayobject.h> #include <omp.h> static PyObject* vadd(PyObject* self, PyObject* args) { PyArrayObject *vec1, *vec2, *vecout; int n; if (!PyArg_ParseTuple(args, "O!O!", &PyArray_Type, &vec1, &PyArray_Type, &vec2)) return NULL; // operand missing if (vec1 == NULL || vec2 == NULL) return NULL; double *v1, *v2, *vout; v1 = (double*)vec1->data; v2 = (double*)vec2->data; //Create a numpy array as an output variable and obtain a pointer vecout = (PyArrayObject *) PyArray_FromDims(1, vec1->dimensions, NPY_DOUBLE); vout = (double*)vecout->data; n = vec1->dimensions[0]; //OpenMP int i; #pragma omp parallel for private(i) firstprivate(n) shared(v1,v2,vout) for (i = 0; i < n; i++) { vout[i] = v1[i] + v2[i]; } return PyArray_Return(vecout); } static PyMethodDef ParallelMethods[] = { {"vadd", vadd, METH_VARARGS, "Add two vectors in parallel"}, {NULL, NULL, 0, NULL} }; PyMODINIT_FUNC initparallelm(void) { (void) Py_InitModule("parallelm", ParallelMethods); // Required call for numpy to work import_array(); }

We need python-dev package installed in our system. Also we need numpy because we are going to parse arrays and vectors with PyArrayObject, this is an object of numpy. The lines 45-50 we have an init function. In this function we specify the name of this module.  The line 49 import utilities for Arrays to work with numpy. 

The lines 36-41 we write all functions in this module in array. In this example we only have a function, vadd.

The lines 5-34 are the vadd function. First we have to parse the arguments in two arrays. We use PyArg_ParseTuple function and PyArrayType to get the vectors. Then we compare the dimmension of two vectors and we compute the sum with OpenMP. Finally we return the result in  PyArray_Return type. Now we can check this module and use it with Python.

$ python setup.py build

$ cd build/lib*

Check with previous example.

In the reference [2] is a complete source code with some math operations parallelized with OpenMP.

References

[1] http://docs.python.org/c-api/index.html
[2] https://bitbucket.org/jjfumero/parallel-math
[3] http://en.wikibooks.org/wiki/Python_Programming/Extending_with_C

Mercurial: some useful commands

The control version system is essential in the software development. Nowadays, I think that people trend to use distributed systems such as mercurial, git or bazaar. These control version systems are more powerfull and complex than centralized systems. In my case I usually work with mercurial. So, I want to share some useful commands or options in mercurial to work faster and safer. 

Create a compressed file

We can create a tgz, zip, tar file from our package or distrubition. Mercurial provides an command for this operation.

$ hg archive -t tgz /tmp/dist.tar.gz

Also we can specify a revision (is selected the tip by default) with -r option.

Changesets between our version and server version

When we have committed our changes, we usually do a pull and an update operations. Before doing these operations, we want to know how many changesets we are going to upload or download. For instance, I have been working for several days and doing some commits. Now, I can upload my changesets to server. It is possible to know how many changesets I am going to upload with outgoing option.

$ hg outgoing

On the contrary is possible too. With incoming option, we know how many changets we will bring.

$ hg incoming

Patching

When we are working with a team, sometimes we do not want to create a new commit because the current copy is not a stable copy. So we need create some temporal changes and share with others as temporal solution. Other typical situation is to resolve a bug. There is at least two ways to do this.

1. The easiest way to do this is apply the import command. For example:

$ hg diff > temporal.patch #### Send temporal.patch $ hg import < temporal.patc

2. With Mercurial Queues.

In my opinion this is a clean way of doing though is more complex. To use mercurial queues is necessary to enable mq as a extensions in out hgrc file.

[extensions] mq =

In the mercurial documentation is very well explained, so I only show an example. We create a new queue with qnew, then we can edit and save all changes that we want. Then, we have to save this patch with qrefresh and export these changes. When we write qpop, the repository return to state before create the qnew. Now, we can apply the patch.

$ hg qnew tutu $ vim main.c   # edit files $ hg qrefresh -m "patch 1 " $ hg export tip > tutu.patch $ hg qpop popping tutu patch queue now empty $ ls files.c  main.c  tata.patch  $ hg qdel tutu $ ls files.c  main.c  tutu.patch $ vim tutu.patch

More information in [1] and [2]. Also, with mq extension is possible change the repository history. In [3] the is a complete guide that shows how to do it.

References

[1] https://developer.mozilla.org/en/Mercurial_Queues
[2] http://stackoverflow.com/questions/4538675/mercurial-patch-creation-and-usage
[3] Change history:  http://mercurial.selenic.com/wiki/EditingHistory
[4] http://hgbook.red-bean.com/read/managing-change-with-mercurial-queues.html