Docker-in-Docker (DinD) capabilities of public runners deactivated. More info

Commit fb3216f7 authored by Betoule Marc's avatar Betoule Marc
Browse files

Add a working plan document for the v1.1

parent 4e3b87ab
#+ V1.1 plan
I see at least before three projects to complete before making the first release:
* The task_id project is not closed:
This is release critical.
- [ ] There is some dark zone (sideeffects):
- how segment without seg_output are treated (no task is stored, what happened when we delete these kind of segs ...)
- Any problem with Orphan tasks ?
- problem for parents giving same str_input outside the special case of groud_by (is this really ok for group_by ?)
- [ ] Does the tracking on disk allow to reconstruct the database
- [ ] I modified the task format to allow for easy glob_seg. It may have break other things (At least the database reconstruction)
- [ ] Is the treatment of redundant tasks resulting from group_by OK
- [ ] The code of multiplex is hard to read/lacks for comments. Variable Names, data structure may be rethought.
* We should bring the default environment to its final state:
This is partly release critical: changes breaking compatibility are
release critical (this is the case of the glob_seg project I think),
API changes for which compatibility could be maintained by
publishing some kind of OldEnvironment classes are not.
- [ ] I started a project (glob_seg/glob_seg_all separation) to make
glob_seg easier to use and more efficient when large number of
task are present (look only in the requested directory). This is not finalize:
- It allows only to glob the direct parent. Going further causes a
big "depth" problem in the current state.
- It is probably buggy with side effects (see what happened on
segment without seg_output.
- Its calling signature should be rethought
- The name should be changed in glob_parents and glob_seg
- [ ] Are we satisfied with seg_input, is this convenient, should we
provide extra function to ease the retrieval of inputs.
- [ ] Are we satisfied with the lst_par, lst_tag, load_param syntax (may become save(name1,name2 ...) and expose(name1,name2...))
- [ ] logged_subprocess is the function I use in every segment,
quick and simple to make it perfect.
* Profiling/optimisation of the code.
This is release critical: At least we need to know whether changes
in the db structure/task object structure/multiplex function ... are
required. The kind of changes that breaks the compatibility.
* depend is not working even the way I want it to work
This is minor for the release but, we should decide what it should do at minima. To me the minimum is:
- [X] Accept that the segment can depend from external files specified as:
This is working in my v1.1
- [ ] Those files should enter in the hash
- [ ] Those files should be stored
* In my branch the only remaining working mode is debug
It seems that manager is not able to serialize the scheduler object anymore
When trying to pickle the s object I am answered that thread.lock object are not pickleable. Strange because lock are here for a while.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment