PLANCK Simulations & Prototyping
The PLANCK Simulation effort is already underway and needs
to be quite extensive. This is a place holder for that effort. The prototyping
effort has not really begun. It is clear that a serious effort will be
required and must begin moderately soon for a number of reasons.
A large simulation effort is required - simulations group exists - see
Many different levels required
Raw Telemetery Stream - full up simulation
Merged data stream
Pixelized data stream
The prototyping effort must primarily be done by the science
team. This is especially true for alogrithm development and instrument
specific items. This presents an issue in that few scientists are trained
to develop fully structured and documented code. One of the roles of the
DPC (and likely in the US a center like IPAC at JPL) would play is to accept
this software and bring it up to standards. This approach will only work
smoothly, if the DPC (& IPAC?) both have the skills and background
and desiminate the standards and help train the scientists to develop code
that can readily be brought into full compliance. This requires some education
on both sides. In an effort as large as the PLANCK processing and analysis
will be, especially given the limited total resources, the full team will
to start with as high a quality near-compliant code to maintain
a reasonable schedule and budget.
Much of the software must be utilized, maintained, and exist
in repositories at the DPCs. This requires close coordination of the scientists
in all countries with the limited resources of the DPCs, particularly when
they are in full production mode.
All major software that can be forseen
Mission Planning tools and commanding aids
Real Time & Quick look software
Instrument Performance Monitoring
Data Stream Merging & Attitude Reconstruction
Systematic Error Analysis Software
Ancillary Data Sets Software
e.g. Beam pattern reconstructions
Calibration and Instrument Signature Removal
Map & Covariance Matrix Generation Software - new alogrithms
Signal Separation algorithms - eg. CMB, SZ, foregrounds, ...
Map correlation algortithms
Power Spectrum Estimation - new alogrithms required
Cosmological Model Test, e.g. statistics of fluctuations, - new alogrithms
Cosmological Parameter Estimation - new alogrithms required
CMB Polarization map and covariance matrix, cosmological parameter estimation
-new alogrithms required
Additional Scientific Goals & Processing
Much new software and potentially very different data structure required
Data Serving Algorithms
Software and Results verification and validation data sets and software
Documentation and Data Base
Data and Documentation transfer - nonimally FINDAS and IDIS but some special
software will be necessary
Distributed Location of Development
Due the large number of institutes and scientists able and
requested to play a role for developing pieces of software needed for the
Planck data pipeline implementation, it appears completely unrealistic
to reduce the number of places where these pieces of work can be done.
It has been demonstrated in space projects, or ground huge collaborations
(see for example the development of software for the LHC or for radio astronomy
throughout the world) to gather contributions from many places. However
for a coherent and efficient development it is crucial to define strict
rules for developing, providing and testing pieces of software under the
control of the persons responsible of implementation work packages and
by software engineers. There are strong arguments for encouraging
the development of activities in this direction.
More generally, one should clearly identify which activities
are relevant to scientific preparation of the mission, which deal with
prototyping of pieces of software or mathematical models for the data processing
and those which are parts of the real data processing incarnation under
the strict control of the DPC Manager. The definition of separate working
groups and putting into place of mechanisms for co-ordinating the assigned
work packages is a first attempt to clarify interfaces and responsibilities.
These concepts are certainly to be iterated and tested
especially during the implementation of the first incarnation of the different
levels of data processing.
This can make for a very unwieldy structure. So, in order
to reduce the number of interfaces and reporting lines, it has been agreed
within the Planck HFI Consortium to group several separate institutions
in single structures (CPAC and LPAC in the UK, POSDAC in France, MPA in
Germany, SSD, IPAC for all US and Canadian contributors).
DEVELOPMENT STRATEGY AND MODEL POLICY
A strong and continued interaction is needed between scientists
and software system engineers. Thus, we need to define a development
philosophy that can encourage and regulate these interactions, taking into
account the different methods and techniques used by different partners.
A high degree of involvement of all scientists is also required, supervised
by the Survey Scientist and the Science Co-ordinator.
Experience has shown in many previous scientific (and
industrial) software projects, that there cannot be too long a time between
the definition of requirements and the implementation of the software itself.
First, because there are unavoidable adjustments of requirements
following scientific and technical improvements of the algorithms.
Second, because too early an implementation will be too
dependent on current computer hardware and on software technology and will
freeze technical evolution.
And finally, because it is well known that long projects
(especially software projects) reduce the motivation of the teams involved.
Two methods can address at least partially these difficulties.
The first is at the heart of the IDIS philosophy: by
defining components in an object-oriented method, we can concentrate our
efforts on the functionality and on the interfaces of the components almost
independently of their exact implementation. The lifetime of a component
can then be much longer than operating system versions or commercial software
The reusability of components, the growing market of
commercial available components and the strong de-coupling between requirements
(or interfaces) done by scientists and implementation by software engineers
are strong arguments for encouraging to develop our activities in this
The second approach uses a method inspired by the RAD
(Rapid Application Development) technology: the final products (the data
processing levels) will be progressively built up through the realization
of consecutive preliminary versions.
In a new version, functionalities are added or improved
by analysing and estimating the performances and defaults of the previous
version. The complete time scale for specifying, designing, implementing
and testing a version cannot exceed 2-3 years.
This approach is very similar to the model philosophy
defined for the instrument.
Each model partly reflects or represents the flight model
and thus implements only partial functionnalities.
Following the general schedule given by ESA for the FIRST/Planck
program, we plan to deliver three versions of the data pipeline, a breadboard
model, a development model and a final model.
Each model design and implementation will be subject
to review by external experts.
Further work in 1999 should produce more precise descriptions
of the exact contents of each model.
Distributed Location of DPC Integration and Operations
Both Planck DPCs are organised in a geographically distributed
fashion for taking the maximum advantage of the expertise available at
the collaboration, without the heavy costs associated with gathering many
experts at two centers only, and sharing of the high costs of data processing
activities among all participating countries.