Simgrid

The "spring cleanups (before the next Big Project kicks in)" release.

Models:
* We can specify the SMPI latency/bandwidth factor with command line
add --cfg=smpi/bw_factor:"threshold0:value0;...;thresholdN:valueN"
or add --cfg=smpi/lat_factor:"threshold0:value0;...;thresholdN:valueN"
You can also use the "config tag" from platform file by adding this line
<prop id="smpi/bw_factor" value="threshold0:value0;...;thresholdN:valueN"></prop>
(see "example/platforms/tag_config.xml" to use "config tag").
Note that the command line supersedes the platform file configuration.
* Change the correction factors used in LMM model, according to
the latest experiments described in INRIA RR-7821.
Accuracy should be improved this way.
* Use the partial invalidation optimization by default for the
network too. Should produce the exact same results, only faster.
* Major cleanup in surf to merge models and split some optimization
mechanisms from the core of the models. As a result you can now
specify which model to use (e.g., --cfg=network/model:LV08
--cfg=cpu/model:Cas01) and which optimization mode to use
(e.g., --cfg=network/optim:lazy --cfg=cpu/optim:TI).
Incompatible combinations should err at initialization. See
--help-models for the list of all models and optimization modes.
* The CLM03 workstation model was dropped for simplicity because it
used the deprecated CM02 network model. Use default instead.
* Rename the TCP_gamma configuration option to network/TCP_gamma
* Rename the coordinates configuration option to
network/coordinates, and document it
* Use now crosstraffic keyword instead of the terribly misleading
fullduplex keyword. It is activated by default now in the current
default model, use --cfg=network/crosstraffic:0 to turn it off.
* Ongoing refactoring the model parsing to make XML files optional
See include/simgrid/platf.h for details (still to be completed)

MSG:
* Major overhaul of the documentation. Almost instructive now :/
* Deprecate the use of m_channel_t mechanism like MSG_task_{get,put}
functions and friends. This interface was considered as
deprecated since over 2 years, it's time to inform our users that it is.
Switch to MSG_task_{send,recv} instead, or compile SimGrid command line
'cmake -Dcustom_flags="-DMSG_USE_DEPRECATED" .' if you really need to
use these (crappy) functions in your code.
These functions will be removed soon. Stop using them now.
* Deprecate MSG_get_host_{table,number}
Implement MSG_hosts_as_dynar() instead.
* Implement MSG_processes_as_dynar() (Closes gforge 13642)
* Remove the public field msg_host_t->name. Use MSG_host_get_name()

Simix:
* Stabilize the parallel execution mode of user contexts
* Introduce configuration variables to control parallel execution:
- contexts/synchro: Synchronization mode to use when running
contexts in parallel (either futex, posix or busy_wait)
- contexts/parallel_threshold: Minimal number of user contexts
that must be part of a scheduling round to switch to parallel
execution mode (raw contexts only)
* Fix bugs that prevented to use suspend/resume along with
synchronization structures.
* Fix bugs in process termination that lead to invalid memory access
in very specific conditions.

SURF:
* Introduce a parallel mode for the models (controlled by surf/nthreads
configuration item). In our tests, running the models in parallel
never lead to any speedups because they are so fast that the gain
of computing each model in parallel does not amortizes the
synchronization costs, even when ultra fast futexes are used.
This is released anyway because YMMV.

SimDag:
* Performance boost by using a swag internally to compute the set of
tasks that are finished and should constitute the return value of
SD_simulate.

SMPI:
* Enable it by default now that it is considered rather stable.

TRACE:
* Documentation of the tracing functions.
* Performance gains when tracing categorized/uncategorized resource
utilization by avoiding calls to get route when updating resource
variables. LMM constraints are being used instead.
* API changed to set task categories. Use MSG_task_set_category instead
of TRACE_msg_set_task_category, and SD_task_set_category instead
of TRACE_sd_set_task_category. They only work if ENABLE_TRACING is ON.
* Bugfix for graphicator, routes not correctly obtained, memory leaks
* Examples for link user variables added (see at examples/msg/tracing/)
* Deprecated function TRACE_msg_set_process_category completely removed
* Trace header updated according to the latest Paje file format
* Tracing network lazy updates, no longer obligate users to use full updates
* --cfg=tracing/platform:1 also registers power/bandwidth variables
* Experimental: let user code declare/set/push/pop application states for hosts
* API changed to allow the manual creation of graph configuration files
for Triva. See TRACE_get_node_types() and TRACE_get_edge_types().

Lua:
* Improve the API of Lua MSG bindings, using the Lua spirit.
* Each simulated process now lives in its own Lua world (globals are
automatically duplicated). It helps writing simulators. Will allow
to run Splay programs within SimGrid in the future.
* Add a Chord example in Lua, equivalent to the MSG one.

MODEL-CHECKING:
* Start the implementation of a solution to express temporal
properties, not only local assertions. This is still an
experimental work in progress, stay clear from it to be safe.

XBT:
* Logs:
- Add new runtime parameters --help-logs and --help-log-categories
to display information about supported logging parameters and
categories.
- Old deprecated parameters --{gras,surf,msg,simix,xbt}-log=...
don't exists anymore.
* Mallocators: allow value NULL for the reset function.
* Dicts:
- New function xbt_dict_new_homogeneous(void(*)(void*)) to
create homogeneous dictionaries, where all the elements share the
same free function. Non homogeneous dictionaries will be
deprecated in the next release.
- Dicts of scalar elements (xbt_dicti_*) are deprecated.
- Multi-level dictionaries are deprecated.
* Dynars:
- new function xbt_dynar_search_or_negative() that is useful when
you have less than 2 million elements in your dynar and don't
want of the extra complexity of catching exceptions when the
element is not found.
* Portability layer
- Make xbt_os_thread module (for thread portability) public.
Documentation is still to come, sorry.
* mmalloc module:
- Cleanups and simplifications to make it maintainable again.
- Exotic features (such as memalign and valloc) were removed.
- The metadata were extended and improved so that the
model-checker becomes able to explore and inspect the heaps.
- This may induce a performance drop when enable_model-checking is
ON in cmake (even if it's not used in the simulation), but it is
necessary at this point to get MC working.

Turn model-checking OFF if simulation performance matters to you.
Not enabling it at runtime is not enough, disable it in cmake.

-- Tue May 15 11:30:19 UTC 2012 Da SimGrid team <simgrid-devellists.gforge.inria.fr>

The "Not coding new stuff allows to polish old things" release.

General
* New bindings to the NS3 packet level simulator (experimental)
* Use the raw (efficient) execution contexts instead of the sysv
(portable) ones when possible.
* libpcre is now mandatory in any cases since not using it led to
severe performance loss and possibly other issues
* Update the XML platforms:
- G5K: include the latest machine in Nancy
- GridPP and LCG: new platforms
* Documentation was partially updated, at least (more to come)

Bug fixes, cosmetics and small improvements
* Free terminated processes before the end of the simulation to avoid
exhausting the memory of users having very dynamic amount of
processes.
* Bug fix and cosmetics about canceling non-running tasks
* Bug fix about the dot loader's issues when using libcgraph

Portability
* Create an installer for windows with nsis (amd64 and win32)
- Add an hello world project to illustrate SimGrid project creation.
- Embed libpcre into the SimGrid installer to avoid
its compilation burden
* The raw execution contexts should work on Apple now
* Port to Windows 64 bits
- Sysv contexts now have an implementation for this arch
- GRAS communication features now support this arch
* Drop support for borland compiler on windows
- this code was not maintained, and we kinda depend on gcc nowadays
* Fix portability issues on kfreebsd/gnu: build error about semaphores
* Fix portability issue on unstable ubuntu: linker became picky on
argument order

-- Wed Oct 5 15:51:01 CEST 2011 Da SimGrid team <simgrid-devellists.gforge.inria.fr>

The "Oops, we broke Macs too" release

Portability
* Fixed contexts detection so that raw ones are used when possible
* On Mac, do not use Posix Ucontexts with gcc v4.[1-5] since this
leads to a strange error, with user code segfaulting sometimes when
the generated code is not perfectly aligned (which is not
controllable from the user side, depends on the amount of code)

XBT
* New macro: CATCH_ANONYMOUS, which is like CATCH(e) but without argument.

-- Mon Jun 27 13:59:03 CEST 2011 Da SimGrid team <simgrid-devellists.gforge.inria.fr>

SimGrid-java (3.6) unstable; urgency=low

* Initial release.
* Split of every thing from SimGrid v3.5 into a separate package.

-- 2011-10-05 Da SimGrid team <simgrid-devellists.gforge.inria.fr>

The Summer Release, also known as the "OMG! They Killed Kenny!" version

Java and Ruby:
* Bindings now constitute their own package, separated from the main one.
Rationale: reduce our maintenance nightmare by reducing the module coupling
They will soon be released on their own on gforge.
* In the meanwhile:
svn co svn://scm.gforge.inria.fr/svn/simgrid/contrib/trunk/simgrid-java
svn co svn://scm.gforge.inria.fr/svn/simgrid/contrib/trunk/simgrid-ruby

GRAS: It is not considered as stable anymore, but experimental. Sorry.
* It's not quite deprecated for now because we have no replacement,
but it may soon become the case.

SMPI
* New MPI functions supported: MPI_Comm_disconnect, MPI_Comm_get_name
* Fortran: New user-level cache variable to store the rank of the running
process. This improves performance by an order of magnitude.
* C: New coccinelle script to automatically locate and modify global and
local static variables.
* Improved SMPI network model with a sender-side gap to account for multiple
parallel sends.

MSG
* New function MSG_comm_get_status(). MSG_comm_test() and MSG_comm_testany()
only say if a communication is finished, no matter whether it succeeded or
failed. You can call MSG_comm_get_status() to know the status of a finished
communication.
* New function MSG_task_dsend() to send a task and detach it. When a
communication is detached, you are never notified of its success or failure
and the memory is released automatically once it is finished. This function
is useful when you don't care about the end nor the success of a
communication.
* Change the prototypes of action replay. Sorry for inconvenience,
but this is really more efficient this way (and to adapt your code,
you just have to fix the initialization, that shouldn't be too long)
* Kill the braindead MSG_task_refcount_dec() function. I guess nobody
ever managed to do anything useful with it.
* New function MSG_comm_testany(). Similarly to MSG_comm_waitany(), it
takes a dynar of communications. It returns immediately and gives the
index of a finished communication (if any).
* New example: a basic implementation of the Chord P2P algorithm.

SURF
* New model for multi-core CPUs. You can now use the core attribute to
precise the number of cores of a host. This is a basic model. Every
process running on the host receives at most the power provided in
the DTD (throughput<=power). Total throughput of process cannot exceed
power * num_cores.
* New peer tag. This peer tag creates a tiny AS comprising a host and a
router linked by an up-link and a down-link (possibly asymmetrical).
This kind of pattern allows to easily build last-mile model style platforms.
Aggregating such patterns in a rule-based AS is thus the technique of
choice for modeling large peer-to-peer/volunteer computing/cloud platforms.
* New model for Vivaldi routing. We transformed the Vivaldi network model
into a Vivaldi routing model (based on the rule-based model). This allows to
combine Vivaldi based latencies with last-mile platforms.

SIMIX
* Added a check for NaN of IEEE754 infinite in the double entries of
the smx_user.c file
* Introduce a new context factory "raw", highly inspirated from the
ucontext factory, but using manually crafted functions in assembly to
do the work in an efficient manner.
* Allow to change the used context factory at run time, not only at
compilation time. Use --cfg=contexts/factory:raw for maximal speed.
* Add an option --cfg=contexts/stacksize:N to set the stack size of the user
contexts at runtime (only with raw contexts or ucontexts).
* Completely rewrote this module to allow parallel execution of user
processes. Use --cfg=contexts/nthreads:N to execute user processes
with N parallel threads (the default is 1, meaning no parallelism).
* Allow to decide dynamically between sequential and parallel modes.
When nthreads > 1, you can use --cfg=contexts/threshold:P to run the user
processes in parallel only when their number is greater than or equal to P
(the default is 2).
* Added a check for NaN of IEEE754 infinite in the double entries of
the smx_user.c file

XBT
* New command line option: if you pass --cfg=verbose-exit:0, SimGrid
won't output the state of processes when interrupted with Ctrl-C
* Add a new function xbt_dynar_to_array that transforms a dynar into a
NULL-terminated array. This may solve backward compatibility issues
due to the change to return type of SD_simulate. See also:
http://lists.gforge.inria.fr/pipermail/simgrid-user/2010-December/002206.html
* Add new macros with variable number of arguments.
- in xbt/log.h: XBT_DEBUG, XBT_VERB, XBT_INFO, etc.
- in xbt/asserts.h: xbt_assert
- in xbt/cunit.h: xbt_test_{add,fail,assert,log}
- in xbt/ex.h: THROWF and RETHROWF.
Define XBT_USE_DEPRECATED if you want to use the old numbered macros like
INFO1, INFO2, etc.
* Change xbt_die() to accept a format string with arguments, just like printf.
* New data structure: xbt_lib_t, like a dict but more general and with better
memory handling.

INSTR
* New configuration options
Options triva/categorized and triva/uncategorized can be used to generate
graph configuration files for Triva visualization tool.
* Configuration option tracing/platform is renamed to tracing/categorized
* XBT logging makes tracing error checks easier, new root log hierarchy: instr
* New TRACE_user_link_variable interface:
User provides the name of the link and the tracing variable to attach to it
* the declaration of tracing categories must be done after the environment
creation
* simpler tracing interface, just one way to declare categories
TRACE_category or TRACE_category_with_color, it is up to you
* links in the trace file are again identified by their names
* trace contains the full platform hierarchy exactly as declared using the ASes
* Options tracing/msg/[task|process]:1 groups the process by hosts
for both cases, tasks and processes must have names that are unique during the simulation
these options generate traces that are suited to gantt-charts, such as the space-time view of Paje
* The experimental option tracing/msg/volume is deprecated
its functionality may be reincorporated if needed
* Buffering
The tracing generates a trace file with unordered timestamped events,
because of the way the core simulator (surf) works. A script available
at the tools directory (fix-paje-trace.sh) can be used to put the events
in order. We have changed the tracing so it can generate ordered timestamped
events in the final trace, but depending on the simulator (and how much time
is simulated) that can lead to a huge memory utilization. It is deactivated
by default, but it can be activated using the --cfg=tracing/buffer:1 switch.

Build Infrastructure
* Define a SIMGRID_VERSION macro in simgrid_config.h.
- We are trying hard to keep the API stable, but it may happen that
some things change (we're a research project after all, not a
nuclear plant operating system). If such things should happen, you
could rely on that macro to adapt.
- current value: 30600 for 3.06.00, aka 3.6
* Define macro MAKE_SIMGRID_VERSION(major, minor, patch) to help building
a number that can be compared with SIMGRID_VERSION.
* Add a build option -Denable_debug (set to ON by default): when set to OFF,
assertions and verbose/debug logging events are disabled at compile time.

-- Tue Jun 21 08:57:43 CEST 2011 Da SimGrid team <simgrid-devellists.gforge.inria.fr>

Model Checking
* New feature to any SimGrid-based simulator: Model-Checking
Check SIN1 for more details.

SMPI
* New Model SMPI (three-interval linear regression for correction factors)
See RR-7426, available at http://hal.inria.fr/inria-00527150
* Ability to use FORTRAN MPI code (through f2c, automatically privatized)
* New MPI functions supported: MPI_Get_count(), MPI_Comm_split()
* New: RAM folding (see RR-7426 and examples/smpi/NAS/DT-folding)
* New: execution sampling (see RR-7426 and examples/smpi/NAS/EP-sampling)
* See also src/smpi/README

Tracing:
Tracing:
* Tracing system
- Tracing API changes: TRACE_start and TRACE_end should not be called
by user-code. They are automatically called by simulators created
with SimDAG, MSG and SMPI if the toolkit is compiled with
tracing_enabled=ON. Categories declaration and utilization remain the
same for MSG and SimDag.
- A function was added to the tracing API to declare categories with
colors:
- TRACE_category_with_color (char *category, char *color)
where color must be in the following format
"%f %f %f", red, green, blue
and red, green, blue are float values in the interval [0, 1]
- User can specify NULL as color parameter, or continue calling
TRACE_category (cat)
On that case, the tracing system will define random colors
- The following command-line options are supported:
--cfg=tracing/filename:msg.trace
--cfg=tracing:1 (activate tracing, needed to use others)
--cfg=tracing/platform:1 (categorized resource use)
--cfg=tracing/uncategorized:1 (uncategorized resource use)
--cfg=tracing/msg/task:1 (task creation)
--cfg=tracing/msg/process:1 (process creation, migration)
--cfg=tracing/msg/volume:1 (volume of MSG send/recv)
--cfg=tracing/smpi:1 (SMPI interface tracing)
--cfg=tracing/simdag:1 (allow SimDAG tasks receive categories)
- examples of examples/msg/tracing updated
* Tracing SimDag
- DAXLoader and DOTLoader functions can generate tasks with categories
- A new function to attribute a category to SD tasks:
TRACE_sd_set_task_category (SD_task_t task, char *category)
* Tracing the MPI interface implemented by SMPI
- Collective operations are traced with states
- Point-to-Point operations are traced with states/links
- Tracing activated by a parameter "-trace filename" passed
to smpirun during execution (considering that simgrid
is compiled with tracing enabled)
- To run the simulation with gdb, the simulator
accepts --cfg=tracing/smpi:1 to trace SMPI
- tesh files to check if smpi tracing is ok
- See examples/smpi/NAS/DT-trace
* GTNetS tracing re-worked
- adaptation to the tracing system of GTNets to cope
with modifications regarding the fullduplex mode
- new tesh files to check if gtnets tracing is ok

MSG
* Asynchronous communications through the functions:
MSG_task_isend/irecv and MSG_comm_test/wait/waitall
* New function: MSG_load_platform_script()
to make possible using a lua script instead of XML files to set up platforms
* New function: MSG_set_function
to associate functions to processes, used when bypassing the parser
* New functions: MSG_task_set_name(), MSG_task_set_compute_duration()

Platforms: Add some more examples in examples/platforms
* Grid'5000: see www.grid5000.fr
* *_30000_hosts.xml: various huge files [mainly scalability testing]

SURF
* Change the XML format. This is a very important modification. SimGrid
3.5 introduces a new hierarchical format based on the notion of
Autonomous Systems. Compatibility with old format is ensured
through the perl script provided in the install bin directory
bin/simgrid_update_xml.
It is now possible to build platforms with specific routing
mechanism (Full/Dijkstra/DijkstraCache/Floyd) and to easily
connect several platforms together. We will try to provide soon
set of realistic platforms exploiting these properties (have a
look at examples/platforms/ for the moment).
* Take the opportunity of the XML format change to be a good XML citizen:
rename link:ctn to link_ctn and similar changes (also dealt with by
simgrid_update_xml)
* Add a new routing scheme (rule-based) using regular expressions. It
enables to have an extremely low memory footprint when the
underlying routing is simple and can be compactly described. You
need to have libpcre4-dev (perl regular expressions) installed if
you want to use this routing scheme.
* Revive the cluster TAG and allow to easily and efficiently (both in
term of memory and speed) connect clusters together. Have a look
at teshsuite/simdag/platforms/ to see how this can be done. With
this tag, you can create clusters with thousands of tasks at no
cost (have a look at examples/platforms/).
Note: clusters are implemented as ASes, so there is no need for an
enclosing AS tag if you have only one cluster in your platform.
* Add new generic functions in the public interface that allows the user
to call SURF 'create_resource' methods from your code (same
functionality as the XML bypass mechanism but with a much lighter
burden).
* Add a new model (enabled through command line --cfg=network/model:SMPI)
that uses a piecewise linear approximation to produce better
results when exchanging small messages.
* Add a new parameter to handle correctly full duplex link and account
for interferences between uplink and downlink communications
(activate with --cfg=fullduplex:1).

SIMDAG
* Rename the SD_READY (all dependencies are satisfied and task is
scheduled) state in SD_RUNNABLE and define a new SD_SCHEDULABLE (all
dependencies are satisfied) state.
This prevents a confusion between the notion of "ready to schedule"
(SD_SCHEDULABLE) used in DAG scheduling and that of "ready to be
simulated" (SD_RUNNABLE) used by the simulation kernel.
* Change the way a task is considered as ready. Instead of removing
dependencies when a task is done, a counter is decreased. This way,
it is always possible to reach ancestors thanks to the
SD_taks_get_parents function (even after the end of the simulation.)
* Change the return type of SD_Simulate from (SD_task_t*) into
xbt_dynar_t. This function was in handling a dynar internally and
converted it into a NULL terminated array for historical reasons.
* New function SD_dotload(char*) to load a DAG described in dot
format. This loader and the corresponding examples require the
installation of the graphviz library.
* Fix a bug in the management of tasks of size 0 in the surf network
models. This problem was only visible with SIMDAG and you should
thus disregard results produced with earlier versions if you
relied on this feature (some tasks were blocked because of this).
* Fix a bunch of stuff that prevented to use classical models with SIMDAG
even though your applications were doing only point-to-point
communications and sequential computations. Now you can really use any
model you want (of course, if you create real parallel tasks, which are
not implemented in most models beside ptaskL07, this will abort).
* Add an example that schedules a DAX on an heterogeneous platform
using a Min-Min strategy.
* New function SD_workstation_get_current_task() that returns the kind
of task currently running on a workstation in the sequential access
mode.
* Raise some warnings when unexecuted tasks remains at the end of the
simulation. This is usually caused by cycles in the DAG.

SIMIX
* New function: SIMIX_process_set_function() called by MSG_set_function
* Change the underlying waiting queue in semaphores so that a process
can wait on several of them simultaneously (as in waitany).
* Fix the way to handle tokens in semaphores so that all access patterns
work: {acquire, acquire_timeout, waitany} / {release, release_forever}.
* kill the dirty pimple SIMIX_message_sizes_output()
Please use (proper) visualization instead

XBT
* New data container: setset (set of sets of elements)
* New module: mmalloc (mapped malloc, allowing to have several
independent segments of malloc)
* New function: xbt_dict_cursor_set_data()
* New functions: xbt_dynar_sort(), xbt_dynar_compare()
* New function: xbt_dynar_is_empty()
* New function: xbt_fifo_get_last_item()
* Fix xbt_dynar_shrink(): use the right element size.
* Fix xbt_dynar_set*(): allow index larger than current size and memset 0
uninitialized areas during expand.
* Fix semaphores: previous implementation was severely broken.
* Use library init/fini functions for our initialization.
- you can use logs and other feature as soon as you want in your
code (even before the xbt_init / MSG_init)
- xbt_exit is now a no-op and produce a warning when used.

GRAS:
* Port GRAS to new SIMIX mechanisms. This allows gras users to
benefit from the latest improvement to the simulation kernel.
* Kill measurement sockets for now. If you rely on them, sorry. This
release is not for you. This feature will be reintroduced in the
future, but we cannot delay the release any further.
* New function: gras_msgtype_get_name().
* Implement gras_agent_spawn in RL too (the prototype changed a bit)
* Fix (at last) the pmm example: it should not randomly fail anymore.

Build chain: bug fixes and overall polishing
* Cmake is now stable enough. Hence, we killed the autotools.
* Port to windows ( TM :)
* Fix the 'make install' target.
No need to use 'make install-simgrid' anymore
* Introduce a 'make dist' target compiling a *source* archive
'make package' compiles a binary archive
* Compile java files only on need
* Add --cd and --setenv command line options to tesh
* Out of source builds are not fully supported yet, but we are close
* Enable supernovae and optimization flags by default for our users

LUA Bindings
* Add layer to set up environment directly from lua, without XML.
* The effect of gras_stub_generator can be achieved through
lua too (check examples/gras/console/ping_generator.lua)

-- Wed, 01 Dec 2010 22:09:23 +0100 Da SimGrid team <simgrid-devellists.gforge.inria.fr>

The "Polishing easter eggs is probably a good idea" release.
This is a bug fixes release only.


Java Bindings
* Fix a bug preventing the tasks from begin garbage collected.

MSG
* Fix a bug occurring when a host involved in a communication fails.
This was not detected properly by the other peer involved in the
communication. Now, it's reported as a network error.

SimDag
* Warn the user about loop dependencies in data flow of DAX files
* Obey the control-flow dependencies of DAX files

Cmake
* Add option "enable_smpi" allowing to not compile SMPI.
Probably useful for the (Mac) users experiencing a build error here
* Improve the detection of lua5.1 and ruby1.8

-- Da SimGrid team <simgrid-devellists.gforge.inria.fr> Tus, 04 May 2010 28 16:11:16 +0100