Commit c4d1d03c authored by Betoule Marc's avatar Betoule Marc
Browse files

starting a modified environment for branch 1.1

parent f09dca85
......@@ -52,6 +52,10 @@ Pipelet is a free framework which helps you :
adapt to different hardware and software architectures.
*** What's new in v1.1
- The glob_seg behavior has been modified for coherence, convenience,
and performance sake. See [[*The%20segment%20environment][The segment environment]].
** Getting started
*** Pipelet installation
**** Dependencies
......@@ -121,7 +125,6 @@ This command ends up with the creation of directory named pipename wich contains
The next section describes those two files in more details.
** Writing Pipes
*** Pipeline architecture
......@@ -348,8 +351,16 @@ The segment code is executed in a specific environment that provides:
2. Functionalities to use the automated hierarchical data storage system.
- =get_data_fn(basename)=: complete the filename with the path to the working directory.
- =glob_seg(seg, regexp)=: return the list of filename in segment seg
working directory matching regexp.
- =glob_seg(seg, regexp)=: Return the list of filename matching the pattern y in the
data directory of parent tasks from the parent segment x.
- =glob_seg_all(seg, regexp)=: Return the list of filename matching
y in the working directory of segment x independantly of
whether the file comes from a task related to the current
task. glob_seg_all is provided to reproduce the behaviour of
old glob_seg for backward compatibility. Its usage should be limited as it:
- potentially breaks the dependancy scheme.
- may hurt performances as all task directories of the segment
x will be searched.
- =get_tmp_fn()=: return a temporary filename.
3. Functionalities to use the automated parameters handling
......@@ -1169,7 +1180,22 @@ And the corresponding cgi script:
#+end_src
* The Pipelet actors
* Contributing to pipelet
** Improvements suggestion
As far as possible, we try to keep pipelet ReadMe driven. If you want
to ask for new features or modifications to pipelet, a good way to do
it is under the form of a patch to the README.org file. If you are
using git:
- Figure out how actually would like pipelet to behave
- Edit the README.org in consequence. Org files are simple text files
that can be conveniently edited using emacs org-mode.
- Commit your proposition with a relevant commit message.
=git commit -a=
- Format the patch for email submission, and send it to us
=git send-email --to HEAD^=
** The Pipelet actors
This section document the code for developers. The code documentation
can be built using the doxygen configuration file
......
......@@ -182,12 +182,33 @@ class Environment(EnvironmentBase):
self.logger.info ("hooking %s"%hook_name)
return self._hook(hook_name, glo)
def glob_seg(self, x, y):
""" Globbing limited to the fatherhood
For unlimited globbing see glob_seg_all.
Parameters
----------
x: string, segment name
y: string, globbing pattern
Returns
-------
Return the list of filename matching the pattern y in the
data directory of parent tasks from the parent segment x.
"""
segx = self._worker.pipe.find_seg(self._worker.task.seg, x)
if segx is None:
self.logger.warning("No parent segment matching %s found"%x)
def glob_seg_all(self, x, y):
""" Return the list of filename matching y in the working
directory of segment x.
Usage of glob_seg_all should be limited:
- potentially breaks the dependancy scheme
- May hurt performances as all task directories of the segment x will be searched
Parameters
----------
x: string, segment name
......@@ -205,7 +226,7 @@ class Environment(EnvironmentBase):
+glob(path.join(self._worker.pipe.get_data_dir(segx),path.join('*/',y)))
def logged_subprocess(self, args, shell=False, except_on_failure=True):
def logged_subprocess(self, args, shell=False, except_on_failure=True, name=None):
""" Execute a subprocess and log its output.
Create files process_name.log and process_name.err
......@@ -221,8 +242,10 @@ class Environment(EnvironmentBase):
tuple (outputfile, errorfile)
"""
proc = args[0]
of = self.get_data_fn(proc+'.log')
ef = self.get_data_fn(proc+'.err')
if name is None:
name = proc
of = self.get_data_fn(name+'.log')
ef = self.get_data_fn(name+'.err')
o = file(of,'w')
e = file(ef,'w')
e.write('#'+' '.join([str(a) for a in args])+'\n')
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment