Papers Related to Media Access Patterns

Here is a list of papers on media access pattern statistics and analysis:

On Feasibility of P2P On-Demand Streaming via Empirical VoD User Behavior Analysis, Distributed Computing Systems Workshops, 2008. ICDCS '08. 28th International Conference on
The Stretched Exponential Distribution of Internet Media Access Patterns, PODC'08
Can Internet Video-on-Demand be Profitable, SIGCOMM'07
GISMO:A Generator of Internet Streaming Media Objects and Workloads, SIGMETRICS'01
Challenges, Design and Analysis of a Large-scale P2P-VoD System, SIGCOMM'08

Error When Running Simulation with Valgrind

When I ran my NS2 simulation with valgrind, I got the following error report:

==25876== Use of uninitialised value of size 8
==25876== at 0x3834AC85C0: profil_counter (in /lib64/tls/libc-2.3.4.so)
==25876== by 0x3834A2E2AF: (within /lib64/tls/libc-2.3.4.so)
==25876== by 0x3835ADAD2F: (within /usr/lib64/libstdc++.so.6.0.3)

It occurs from time to time. It seems that only happens when running on a 64 bit system.

I think the possible reason is the simulator is compiled with flag -pg because the field "profil_counter" is used by gcc for profiling. Got no solution yet.

Youdao-DicticibaWikipediaWictionaryChambers (UK)Google imagesGoogle defineThe Free DictionaryJoin exampleWordNetGoogleAnswers.comrhymezone.comMerriam-Webster

The Importance of not Using "bind()" in NS2

As described in my last post, I used valgrind to find out the memory usage of my simulator. Finally, I found that a invocation of "bind()" in UdpAgent::UdpAgent()" consumes most memory. That reminds me that the ns2 manual recommends to use dynamic binding instead of "bind" when the object of a class is frequently created. In my case, the P2P application requires two UdpAgents for a connection. Since the connections are created and removed constantly, the memory consumed by bind() keeps increasing. To solve this problem, I modified class UdpAgent as follows:

apps/udp.h
add two declarations:
virtual void delay_bind_init_all();
virtual int delay_bind_dispatch(const char *varName, const char *localName, TclObject *tracer);
apps/udp.cc
delete "bind("packetSize_", &size_);" in UdpAgent::UdpAgent() and UdpAgent::UdpAgent(packet_t type)
add two definition:
void UdpAgent::delay_bind_init_all()
{
delay_bind_init_one("packetSize_");
Agent::delay_bind_init_all();
}

int UdpAgent::delay_bind_dispatch(const char *varName, const char *localName, TclObject *tracer)
{
if (delay_bind(varName, localName, "packetSize_", &size_, tracer))
return TCL_OK;
return Agent::delay_bind_dispatch(varName, localName, tracer);
}

Then type "make clean; make" to rebuild ns2. Now the memory usage and simulation time is reduced significantly.

I'm not very clear about what does it do when "bind()" is invoked and why the memory is not released even when the UdpAgent object is deleted. Anyone who knows the answer please post me a message.

Profiling Memory Usage for NS2 Simulation

I am using ns2 to simulate peer-to-peer networking. However, it turns out that the simulator consumes much more memory than I expected. I am sure that my algorithm could not use so much memory. What's happening during the simulation.

I tried dmalloc, but it seems that the dmalloc supporting in ns2 is out-of-data and causes some error. Finally I gave up dmalloc and turned to valgrind.

I used valgrind to detect memory leaks and errors. Today I learned that it can also do memory usage profiling.

First, you have to pass '-g' option to compiler and re-make ns2. It is recommended to remove any optimization flags when debugging.

Then run your program by typing "valgrind --tool=massif ns test_script.tcl". BTW, I recommend u to specify a threshold for massif by using "valgrind --tool=massif --threshold=0.01 ns test_script.tcl" to get more detailed information.

massif will generate a report in current directory. The default name is massif.out.pid, where pid is the process number.

massif.out is a plain text file. Though u can read it directly, there is another helpful tool, ms_print to gather information in the report and print it more readable. Use ms_print --threshold=0.01 massif.out to format the report.

The output of ms_print is very easy to understand and the online document of valgrind has a detailed description about that. Please refer to http://valgrind.org/docs/manual/ms-manual.html

Enable DMalloc support for NS2

Since my NS2 eats too much memory, which makes the simulation not scalable, I was trying to profile the memory usage by enable dmalloc support for NS2.

Originally I download ns-2.33-all-in-one and install it on my Ubuntu 8.04 Linux.

The NS2 manual seems to show that it very easy to enable dmalloc for NS2, but it's kinda out of date.

First, ns2 needs an old version of dmalloc, for example, I use dmalloc-4.8.2. If u use dmalloc-5.5.1, compiler will complain "_malloc_leap was not declared in this scope"

Extract and drop dmalloc-4.8.2 in the parallel directory of ns-2.33, build it using "./configure" and "make". "make install" is unneccessary. If you use "make install" to install dmalloc in your system, the configure script of ns-2.33 will complain "checking for return.h... no".

Change working directory to ns-2.33, type " ./configure --with-otcl=../otcl-1.13 --with-tclcl=../tclcl-1.19 --with-dmalloc=../dmalloc-4.8.2", it will generate Makefile for you.

Try "make" in ns-2.33, a link error occur: multiple definition of TclpAlloc. This is because libtcl8.4.a already defines TclpAlloc and ns-2.33/lib/dmalloc_support.cc tries to redefine it. I actually have no idea of how to solve this problem. A temporary solution is, if u do not care about the memory TclpAlloc and Tcl_Alloc, just comments out the definition of these two funtions in dmalloc_support.cc and then compilation will success.

After building ns2 with dmalloc, then u can follow the steps in ns2 manual except for the one adding alias for bash. It seems to be a typing error in my ns2 manual. The correct alias should be: function dmalloc { eval `command dmalloc -b $*`; }

That's all.

Make it Inline

When compiling source code using g++, the -O3 flags turns on -finline-functions, which tells g++ to integrate all simple functions int to their callers. If no optimization flags are specified, the default one is -O0, which does no optimization.

The default in g++ seems to be to ignore "inline"
unless one gives the options -finline-functions

references:
http://gcc.gnu.org/onlinedocs/gcc-3.4.2/gcc/Optimize-Options.html#Optimize-Options

Using gprof with NS2

gprof is a powerful tool to measure the running time of each function in a program, which is widely used to analyze and speed up programs.

To use gprof, a program should be BOTH compiled and linked with -pg, -g is also needed for compilation. For example, given a source file hello.c, it should be compiled using gcc -g -pg hello.c -o hello

In NS2, since the compilation process is done using Makefile, which specifies compilation options and link options in a separate form, two modifications should be done to the Makefile as follows:

CCOPT = -Wall -g -pg
LDFLAGS = -Wl,-export-dynamic -pg


Type "make clean & make" to regenerate executable file. Then run your simulation as usual. When ns2 exits, a new file "gmon.out is created. You can use gprof NS_BIN_DIR/ns gmon.out to see the results, where NS_BIN_DIR denotes the directory that the executable file "ns" is located.

How to compile NS2 on 64-bit System

When compiling NS-2.33 on a 64-bit CPU system, I got following error message from linker:
"skipping incompatible libXext.so when searching for -lXext"
cannot found libXext

The reason is the default directory in ns2 to look for libXext is /usr/X11R6/lib, where the 32-bit libXext is located. On a 63-bit system, we should change the directory to /usr/X11R6/lib64/. This can be done with a single command:

grep -rl "/usr/X11R6/lib" * | xargs sed -i -e 's/\/usr\/X11R6\/lib/\/usr\/X11R6\/lib64/'

Change current dir to ns-all-in-one-2.33 and execute the aboe command and run ./install. Then ns2 will be successfully compiled.

NOTE: the command should be run only once, otherwise there would be errors.