Skip to content
Snippets Groups Projects
  1. Jun 15, 2020
    • Lionel GUEZ's avatar
      Create `loop_shp_tr_append.py` · c58c21b7
      Lionel GUEZ authored
      Replace `loop_shp_append.py` by `loop_shp_tr_append.py`. Instead of
      looping on years, we assume we are working on a single year. We could
      call this script inside a loop on years in the shell. The advantage is
      that we need to make fewer assumptions on the arborescence of
      files. Also, we can make `loop_shp_tr_append.py` more general by
      starting and ending at arbitrary days. We drop the timing that was in
      `loop_shp_append.py`. We add the creation of file `ishape_last.txt`.
      
      In script `shp_tr_append.sh`, assume that the directories contain
      files `ishape_last.txt` and modify `ishape_last.txt` in the
      destination directory.
      
      In program `test_overlap`, read grid description from a file
      `grid_nml.txt` in SHP triplet directory, instead of from standard
      input. This is the same modification we made for program
      `eddy_graph`. Modify input files accordingly.
      c58c21b7
  2. Jun 10, 2020
  3. Jun 03, 2020
    • Lionel GUEZ's avatar
      Replace `mkdir_nproc.sh` by `eddy_graph.sh` · fe8820af
      Lionel GUEZ authored
      Replace script `mkdir_nproc.sh` by script
      `eddy_graph.sh`. `eddy_graph.sh` is a wrapper script for program
      `eddy_graph`. It includes the creation of output directories that was
      done by `mkdir_nproc.sh`. `mkdir_nproc.sh` was only used for program
      `eddy_graph`. There was the need to automate the concatenation of
      output files for a run with several MPI processes.
      
      Add a test for `eddy_graph`.
      
      Improve the handling of the NetCDF Fortran library in cmake files.
      Add the configuration of `eddy_graph_in.sh`.
      
      In main program `eddy_graph`, do not write title lines in files
      `unit_number_eddies` and `unit_edgelist` because these files are
      concatenated for all the MPI processes by `eddy_graph.sh`. So the
      title lines are added by `eddy_graph.sh`.
      fe8820af
  4. May 28, 2020
    • Lionel GUEZ's avatar
      Add test for `eddy_graph` · f72c0cfe
      Lionel GUEZ authored
      In script `plot_snapshot.py`, add dummy arguments to function snapshot
      so that it does not use global variables and it can work after the
      module is reloaded. Bug fix for the suptitle in function snapshot.
      
      Add test for program `eddy_graph`. Add the corresponding input NetCDF
      files and the corresponding dates in
      `Tests/Input/Region_2/SHP_triplet`.
      f72c0cfe
  5. May 19, 2020
    • Lionel GUEZ's avatar
      Align dates in `read_eddy_graph.py` · 6d1f9eea
      Lionel GUEZ authored
      In script `read_eddy_graph.py`, create a pygraphviz graph and write
      the corresponding file instead of directly plotting with
      Matplotlib. Also, create subgraphs in the pygraphviz
      graph. Motivation: aligning nodes with the same date. Moreover: it is
      more convenient to visualize a graphviz file. Moreover, networkx
      documentation seems to advise against relying on networkx's own
      visualizing functionality.
      6d1f9eea
    • Lionel GUEZ's avatar
      New test `Eddy_graph_md2` · 5b75c1c7
      Lionel GUEZ authored
      In script `plot_snapshot.py`, add title with date index.
      
      Rename `read_overlap.py` to `read_eddy_graph.py`.
      
      Rename `loop_shpcat.py` to `loop_shp_append.py` and `shp_tr_cat.sh` to
      `shp_tr_append.sh` because an analogy to the cat command is confusing:
      the cat command does not modify its arguments and the result is in the
      order of the arguments.
      
      Import small NetCDF files in `Tests/Input/Region_1`.
      
      Bug fix in `short_tests.json`: missing mpiexec for `test_read_eddy`.
      
      New test `Eddy_graph_md2`, with corresponding input files.
      5b75c1c7
  6. May 14, 2020
  7. Apr 28, 2020
    • GUEZ Lionel's avatar
      Added first test of program `eddy_graph` · e476a644
      GUEZ Lionel authored
      Added a trick to verify at the beginning of execution of program
      `extraction_eddies` that the directory `SHP_triplet` exists.
      
      Added a script to created as many directories as MPI processes, for
      program `eddy_graph`.
      
      Added stdout keys to JSON tests, so that the stdout files are not
      named `mpiexec_stdout.txt`.
      
      Added first test of program `eddy_graph`.
      e476a644
    • GUEZ Lionel's avatar
      Trim names of files and directories · 504bb737
      GUEZ Lionel authored
      Bug fixes in main program `eddy_graph`. Add default value for
      max_delta. Trim names of files and directories.
      
      Following upgrade of `test_compare.py`, remove `mkdir_run.sh` and
      write the mkdir command in the JSON tests files.
      504bb737
  8. Apr 27, 2020
  9. Apr 16, 2020
  10. Apr 15, 2020
    • GUEZ Lionel's avatar
      Use `shp_triplet` components for field identifiers · 59b5a302
      GUEZ Lionel authored
      Use the components for field identifiers of a variable of type
      `shp_triplet` instead of storing these field identifiers in variables
      of modules `init_shapefiles_m` and `read_field_indices_m`. The dummy
      argument hshp of procedure `read_field_indices` becomes
      `intent(inout)`.
      59b5a302
    • GUEZ Lionel's avatar
      Create derived type `shp_triplet` · df32aa8f
      GUEZ Lionel authored
      Create derived type `shp_triplet`. Motivation: we need to manipulate
      two triplets of shapefiles in program `eddy_graph`. Use everywhere a
      variable of type `shp_triplet` instead of the three variables
      `hshp_extremum`, `hshp_outermost`, `hshp_max_speed` when all three of
      them appear. Rename character variable `shp_triplet` to
      `shp_triplet_dir`.
      df32aa8f
    • GUEZ Lionel's avatar
      Polish · 35561387
      GUEZ Lionel authored
  11. Apr 08, 2020
    • GUEZ Lionel's avatar
      `test_get_snapshot` to `test_get_dispatch_snap` · fda1e6cd
      GUEZ Lionel authored
      In procedure `dispatch_snapshot`, rename dummy argument m to rank,
      clearer.
      
      Rename program `test_get_snapshot` to `test_get_dispatch_snap` and
      make it call `dispatch_snapshot` instead of `send_snapshot`. The only
      difference in result is that the program now writes files
      `isolated_nodes.txt` and `number_eddies.csv`.
      
      Check in program `test_overlap` that it is run on a single MPI process.
      fda1e6cd
  12. Apr 07, 2020
    • GUEZ Lionel's avatar
      Run MPI programs with mpiexec · d89d6b90
      GUEZ Lionel authored
      Change order of dummy arguments of procedure `get_snapshot` to make it
      more convenient to call with keywords. Rename dummy argument m to
      rank, clearer.
      
      Bug fix in `short_tests_.json`: run `test_read_snapshot` and
      `test_overlap` with `mpiexec -n 1`, else the programs may not be able
      to read standard input.
      d89d6b90
    • Lionel GUEZ's avatar
      Add scripts to compare with Matlab output · 8b804de2
      Lionel GUEZ authored
      Add scripts to compare with Matlab output and script to concatenate shapefiles.
      8b804de2
  13. Mar 06, 2020
    • Lionel GUEZ's avatar
      Add a receive test for program `test_get_snapshot` · d07eaadf
      Lionel GUEZ authored
      On our way to a generalization of script `filter.py`: use a function
      to define the filter.
      
      In script `plot_snapshot.py`, accept a triplet of shapefiles without
      `ishape_last.txt`.
      
      Add a test for program `test_get_snapshot` where one of the processes
      does a reception in procedure `get_snapshot`.
      
      Bug fix in main unit `test_get_snapshot`: we send and receive date k.
      d07eaadf
    • Lionel GUEZ's avatar
      Four dates in `Input/Region_1/SHP_triplet_noise` · c0fbe3c3
      Lionel GUEZ authored
      Add script for concatenation of triplets of shapefiles.
      
      Generalize script `plot_snapshot.py` to read a single date in a
      triplet of shapefiles containing several dates.
      
      Bug fix in `Tests/CMakeLists.txt`: missing source file.
      
      Include four dates in `Tests/Input/Region_1/SHP_triplet_noise`, in
      preparation for a test with `test_get_snapshot`. Main unit
      `test_nearby_extr` must then read the number of shapes for the first
      date in `ishape_last.txt`.
      c0fbe3c3
  14. Mar 05, 2020
    • Lionel GUEZ's avatar
      Generalize main unit `test_get_snapshot` · e1aa5bcc
      Lionel GUEZ authored
      Prepare for a run of `test_get_snapshot` which receives a
      snapshot. Generalize main unit `test_get_snapshot`: it is now
      parallelized; it reads a date index from stdin; it sends a snapshot
      from process 1 to process 0 if it is run with 2 processes.
      
      The logic of the overlap program will be that all processes access the
      triplet of input shapefiles. So modify subroutine
      `read_field_indices`: process 0 reads field indices and broadcasts
      them. This implies that programs `test_overlap`, `test_read_snapshot`
      and `test_read_eddy` use MPI.
      
      Generalize `algorithm.py`.
      
      In program `test_send_recv`, read input SHP triplet from command line
      instead of requiring it to be present in the current directory with
      the name `SHP_triplet_old`. Generalize `test_send_recv`: read
      `ishape_last` from file. In `test_send_recv`, all processes must now
      access the triplet of shapefiles since process 0 reads field
      indices. For the sake of uniformity, read and broadcast k1 from
      process 0.
      e1aa5bcc
  15. Feb 18, 2020
    • Lionel GUEZ's avatar
      First compilation and test of `get_snapshot` · 29ee10af
      Lionel GUEZ authored
      First compilation and test of procedure `get_snapshot`. Change order
      of dummy arguments of procedure `get_snapshot`: make it more
      convenient for call with keywords. Replace dummy arguments periodic
      and `dist_lim` by dummy argument copy. Add dummy arguments
      `ishape_last` and k1. Promote `hshp_extremum`, `hshp_outermost`,
      `hshp_max_speed` from local variables to dummy arguments.
      
      In program `test_read_snapshot`, read input SHP triplet from command
      line instead of requiring it to be present in the current directory
      with the name `SHP_triplet_old`.
      
      Remove directory `Tests/Input/Region_1/SHP_triplet`, which was not used.
      29ee10af
    • Lionel GUEZ's avatar
      Make date index `intent(in)` in `read_snapshot` · 7e9c066b
      Lionel GUEZ authored
      Change of idea: make date index an intent in argument of procedure
      `read_snapshot`. Replace dummy arguments `ishape_first` and scalar
      `ishape_last` by k1 and array `ishape_last`. It is now the task of
      `read_snapshot` to find the ishapes to read. This seems clearer.
      7e9c066b
    • Lionel GUEZ's avatar
      Replace `contents.csv` by `ishape_last.txt` · 67346265
      Lionel GUEZ authored
      Change of idea: instead of storing `date_index` and `ishape_last` in a
      file `contents.csv`, store only `ishape_last` in a file
      `ishape_last.txt`. The `date_index` column was useless and it
      complicated reading the file. The first value of `date_index` can be
      read from one of the three DBF files. Also, to roll with the flow, let
      the lower bound of `ishape_last` in main unit `test_overlap` be 1
      instead of k1.
      
      Change order of actual arguments of `dbf_read_attribute_03` following
      revision of `Shapelib_03`.
      67346265
  16. Feb 07, 2020
    • Lionel GUEZ's avatar
      Bug fix: rank of MPI processes is 0-based · bf3c1053
      Lionel GUEZ authored
      Bug fix in procedure `dispatch_snapshot`: the rank of MPI processes is 0-based.
      
      Bug fix: remove `extr_map_outermost.nc`, which was not used since
      commit ad420d25.
      bf3c1053
    • Lionel GUEZ's avatar
      Use SHP triplets with several snapshots · d023fd3f
      Lionel GUEZ authored
      Organize better the directory `Input`: create sub-directories for regions.
      
      Big change: use SHP triplets that can contain several snapshots. So
      program `test_overlap` reads a single SHP triplet. The dates that
      `test_overlap` should read are now specified in the input
      namelist. SHP triplet directory must now contain a file named
      `contents.csv`. `test_overlap` reads `contents.csv` into variables k1
      and `ishape_last`. So there is no longer any need to read the number
      of entities in one of the shapefiles.
      
      For each region, concatenate SHP triplets for dates 2006/01/01 and
      2006/01/02.
      
      Read namelist from process 0 in program `test_send_recv`: it is
      simpler since it is the default setting of mpiexec and I do not see
      any drawback.
      
      In programs `test_read_snapshot` and `test_send_recv`, just hard code
      for now the ishapes we want to read in the SHP triplet.
      d023fd3f
  17. Jan 28, 2020
    • Lionel GUEZ's avatar
      Generalize `read_snapshot` · d18ad095
      Lionel GUEZ authored
      Generalize `read_snapshot`. Add dummy argments `ishape_first` and
      `ishape_last`. This allows getting a snaphot from a triplet of
      shapefiles containing several snapshots. Consequence: move the call to
      `shp_get_info_03` from `read_snapshot` to `test_overlap`,
      `test_read_snapshot` and `test_send_recv`.
      
      Remove opening and closing of shapefiles from procedure
      `get_snapshot`. Since a triplet of shapefiles can contain several
      snapshots, we do not want to open and close the same shapefiles
      several times from the same MPI process.
  18. Jan 10, 2020
    • Lionel GUEZ's avatar
      Complete `send_snapshot` and `recv_snapshot` · 9fb56f5f
      Lionel GUEZ authored
      Pass argument copy to `read_snapshot` rather than arguments periodic
      and `dist_lim`. Motivation: do not recompute copy at each call of
      `read_snapshot`. `read_snapshot` is going to be called many times in
      overlap program, with always the same value of copy. And we do not
      need the values of periodic and `dist_lim` separately in
      `read_snapshot`.
      
      Complete `send_snapshot` and `recv_snapshot` by sending and receiving
      an entire snapshot. So we need nlon, nlat and copy in `recv_snapshot`
      to allocate `extr_map`. It is better to pass arguments nlon, nlat and
      copy than sending and receiving the shape of `extr_map` with MPI, of
      course, since the shape is the same for all snapshots. In program
      `test_send_recv`, write the entire received snaphot.
      
      Bug fix in `Tests/CMakeLists.txt`: several targets missed
      `${netcdff_INCLUDE_DIR}`.
      
      Create procedure `write_snapshot` from code in main program
      `test_read_snapshot`, because we need to do this also in program
      `test_send_recv`. Note the dissymmetry with `read_snapshot`, which
      does not include the opening and closing of the shapefiles. That is
      because we anticipate calling `read_snapshot` several times with the
      same shapefiles in program overlap.
      
      Change order of dummy arguments of procedure `write_extr_map`. More
      convenient for calling it with keywords from `write_snapshot`.
      9fb56f5f
    • Lionel GUEZ's avatar
      Simplify cmake files · 4ca261bb
      Lionel GUEZ authored
      Simplify cmake files, taking advantage of improved packaging of
      libraries. This also much simplifies the configuration process: the
      user typically has only two directories to specify: `CMAKE_PREFIX_PATH` and
      `fortrangis_INCLUDE_DIR`. Also `Shapelib_03` no longer depends on EZMPI,
      since revision 129a0cc.
      
      Add test for `test_send_recv`.
      4ca261bb
  19. Dec 26, 2019
    • Lionel GUEZ's avatar
      Send and receive a whole eddy in `test_send_recv` · 0b01c42f
      Lionel GUEZ authored
      We need the dummy version of EZMPI for sequential programs linking
      with `Shapelib_03`.
      
      Replace `include_directories` by `target_include_directories` in
      `Tests/CMakeLists.txt`. Motivation: some targets do no use ezmpi. This
      implies adding directories to `target_include_directories` for
      `extraction_eddies`.
      
      In `send_snapshot` and `recv_snapshot`, send and receive a whole
      eddy. Read and write a snapshot in `test_send_recv`.
      0b01c42f
  20. Dec 19, 2019
    • Lionel GUEZ's avatar
      Use library EZMPI · a9f5fa95
      Lionel GUEZ authored
      In procedures `recv_snapshot` and `send_snapshot`, use library EZMPI
      instead of MPI directly.
      
      Rename `receive_snapshot` to `recv_snapshot` for consistency with MPI
      calls. In procedure `send_snapshot`, invert order of dummy arguments
      tag and dest for consistency with MPI calls.
      a9f5fa95
  21. Dec 18, 2019
    • Lionel GUEZ's avatar
      Begin `receive_snapshot` and `send_snapshot` · f2cb5757
      Lionel GUEZ authored
      Fill in procedures `receive_snapshot` and `send_snapshot`. This is
      just a beginning with part of an eddy. Add test program.
      f2cb5757
    • Lionel GUEZ's avatar
      Improve `get_snapshot` · 9adf5c7a
      Lionel GUEZ authored
      hsp arguments in procedure `read_snapshot` can be declared with
      `intent(in)` since they are `intent(in)` in `read_eddy` since commit d815bf76.
      
      Bug fixes in procedure `get_snapshot`: remove duplicated declaration
      of nlon, nlat; add argument `dist_lim`, which must be passed to `read_snapshot`.
      
      In procedure `get_snapshot`, read the triplet of shapefiles from a
      directory named with the date index (for the now moment, we will see
      later for something more efficient). Check that the date index in the
      shapefiles is the date index required.
      
      In procedure `get_var`, take advantage of new procedure `nf95_get_missing`.
      9adf5c7a
  22. Nov 20, 2019
Loading