MAR-tools issueshttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues2023-12-07T14:35:52+01:00https://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/75Fix: pbs run script generation2023-12-07T14:35:52+01:00Patryk KiepasFix: pbs run script generation* `epilogue.sh` could be in shared
* `run_chunk.sh` must `cd` into chunk directory and the `shared/` folder* `epilogue.sh` could be in shared
* `run_chunk.sh` must `cd` into chunk directory and the `shared/` folderPatryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/74Run mar-tools using the restart files2023-11-21T11:01:49+01:00Patryk KiepasRun mar-tools using the restart filesPatryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/73Generate MAR source code normally generated by NESTOR2023-11-21T10:53:42+01:00Patryk KiepasGenerate MAR source code normally generated by NESTORThis includes:
* `MARdim.inc`
* `mardim_mod.f90`
* `MAR_SV.inc`
* `mar_sv_mod.f90`This includes:
* `MARdim.inc`
* `mardim_mod.f90`
* `MAR_SV.inc`
* `mar_sv_mod.f90`Patryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/70MAR test should mimick "mar build/mar run"2023-03-30T18:54:29+02:00Patryk KiepasMAR test should mimick "mar build/mar run"Instead of using tags, we should be able to run MAR test like this:
```shell
# With remote repository
mar test --machine=<MACHINE_CONF> --experiment=<EXPERIMENT_CONF> --expecution-nestor=<EXECUTION_CONF> --expecution-mar=<EXECUTION_CONF...Instead of using tags, we should be able to run MAR test like this:
```shell
# With remote repository
mar test --machine=<MACHINE_CONF> --experiment=<EXPERIMENT_CONF> --expecution-nestor=<EXECUTION_CONF> --expecution-mar=<EXECUTION_CONF> [--only-build] ... commit url etc.
# Or local changes
mar test <PATH_TO_REPO> ....
```Patryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/68Analyse cmd files from vanilla-MAR run scripts2023-03-21T17:07:13+01:00Patryk KiepasAnalyse cmd files from vanilla-MAR run scripts[Normal_Run_MAR_ANb01.15.02.01-28.cmd](/uploads/700b72f2290dccaff113044c9c5e5dab/Normal_Run_MAR_ANb01.15.02.01-28.cmd)
[Restart_MAR_ANb01.18.12.01-31.cmd](/uploads/979191f7668baf805e0d06c190b4113e/Restart_MAR_ANb01.18.12.01-31.cmd)
Che...[Normal_Run_MAR_ANb01.15.02.01-28.cmd](/uploads/700b72f2290dccaff113044c9c5e5dab/Normal_Run_MAR_ANb01.15.02.01-28.cmd)
[Restart_MAR_ANb01.18.12.01-31.cmd](/uploads/979191f7668baf805e0d06c190b4113e/Restart_MAR_ANb01.18.12.01-31.cmd)
Check restart files: `/gpfsstore/rech/brn/commun/b01_restart_201812/`Version 0.4Patryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/65Mechanism for storing results in the $STORE after a successful stage2023-03-02T13:54:40+01:00Patryk KiepasMechanism for storing results in the $STORE after a successful stageUnforseen futurePatryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/64Keep and verify additional information about each stage (e.g. configuration u...2023-03-02T10:37:27+01:00Patryk KiepasKeep and verify additional information about each stage (e.g. configuration used, compiler options, etc.)Version 0.4Patryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/60Error during mar build command on JeanZay while using cache2023-03-01T13:44:55+01:00Patryk KiepasError during mar build command on JeanZay while using cache```
[100%] Linking Fortran executable bin/NESTOR
[100%] Built target NESTOR
Checking cached files...
Traceback (most recent call last):
File "/linkhome/rech/genlsc01/rlmd343/.conda/envs/abo/bin/mar", line 8, in <module>
sys.exit(ma...```
[100%] Linking Fortran executable bin/NESTOR
[100%] Built target NESTOR
Checking cached files...
Traceback (most recent call last):
File "/linkhome/rech/genlsc01/rlmd343/.conda/envs/abo/bin/mar", line 8, in <module>
sys.exit(main())
File "/linkhome/rech/genlsc01/rlmd343/.conda/envs/abo/lib/python3.10/site-packages/mar_tools/mar.py", line 27, in main
args.func(args)
File "/linkhome/rech/genlsc01/rlmd343/.conda/envs/abo/lib/python3.10/site-packages/mar_build/cmd/build.py", line 47, in build
nestor_run.perform(
File "/linkhome/rech/genlsc01/rlmd343/.conda/envs/abo/lib/python3.10/site-packages/mar_build/stages/nestor_run.py", line 51, in perform
file_cache.watch_file(Path(nestor_run_dir, "LSCfil.dat"))
File "/linkhome/rech/genlsc01/rlmd343/.conda/envs/abo/lib/python3.10/site-packages/mar_tools/common/cache.py", line 47, in watch_file
self._hashes[str(path_file)] = self._compute_file_md5(path_file)
File "/linkhome/rech/genlsc01/rlmd343/.conda/envs/abo/lib/python3.10/site-packages/mar_tools/common/cache.py", line 68, in _compute_file_md5
with open(path_file) as f, mmap(f.fileno(), 0, access=ACCESS_READ) as fd:
ValueError: cannot mmap an empty file
```Version 0.4Patryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/58Prepare 2 days run with debug and no optimisations (use these options `-g -O0...2023-03-01T10:57:21+01:00Patryk KiepasPrepare 2 days run with debug and no optimisations (use these options `-g -O0 -w -init=zero -init=arrays -static -vec_report0 -mp1 -check all -traceback`)Version 0.4https://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/56Compare results of reference MAR run by Cecile2023-03-01T10:57:34+01:00Patryk KiepasCompare results of reference MAR run by CecileFiles are stored here: `/gpfsstore/rech/brn/commun/ANT35_ERA5_176x148x24_201502`.Files are stored here: `/gpfsstore/rech/brn/commun/ANT35_ERA5_176x148x24_201502`.Version 0.4Patryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/55Test why INTEL compiled MAR doesn't requires these files: `ONEstation.dat`, `...2024-01-11T13:17:05+01:00Patryk KiepasTest why INTEL compiled MAR doesn't requires these files: `ONEstation.dat`, `ICEvou.dat`, `ONEvou.dat`Use `-O0` for the test run and check if this is due to some optimizations!
This is connected to: #48Use `-O0` for the test run and check if this is due to some optimizations!
This is connected to: #48Version 0.4Patryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/54MAR logs function for preparing info-package after an error2023-02-07T10:13:02+01:00Patryk KiepasMAR logs function for preparing info-package after an error`mar logs <OUTPUT_DIR>` takes all the important information about the current execution and creates a debugging report.`mar logs <OUTPUT_DIR>` takes all the important information about the current execution and creates a debugging report.Unforseen futurePatryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/42On obelix, the built-in cmake does not activate2023-02-01T13:11:45+01:00Patryk KiepasOn obelix, the built-in cmake does not activateVersion 0.4Patryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/40New handling of mar-tools directories on $SCRATCH/$HOME/$WORK etc.2023-02-01T12:26:32+01:00Patryk KiepasNew handling of mar-tools directories on $SCRATCH/$HOME/$WORK etc.To think about !To think about !Unforseen futurePatryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/36Use joblib for cache'ing some of the computations (e.g. running NESTOR)?2023-02-01T10:20:51+01:00Patryk KiepasUse joblib for cache'ing some of the computations (e.g. running NESTOR)?https://github.com/joblib/joblib/https://github.com/joblib/joblib/Unforseen futurePatryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/35Improve compiler selection with direct use of CMake instead of env vars2023-01-31T18:57:40+01:00Patryk KiepasImprove compiler selection with direct use of CMake instead of env varsIf it is only possible! This could massively simplify the CMake compilation process and ease the integration of it directly in the MAR source code (one day maybe!).If it is only possible! This could massively simplify the CMake compilation process and ease the integration of it directly in the MAR source code (one day maybe!).Unforseen futurePatryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/32Unused include paths in the JZ GNU configuration2023-01-30T14:41:51+01:00Patryk KiepasUnused include paths in the JZ GNU configurationWhile compiling NESTOR and MAR:
```
[ 6%] Building Fortran object CMakeFiles/NESTOR.dir/src/CHKcel.f.o
f951: Warning: Nonexistent include directory '/gpfslocalsys/slurm/current/lib/include' [-Wmissing-include-dirs]
```
Investigate the...While compiling NESTOR and MAR:
```
[ 6%] Building Fortran object CMakeFiles/NESTOR.dir/src/CHKcel.f.o
f951: Warning: Nonexistent include directory '/gpfslocalsys/slurm/current/lib/include' [-Wmissing-include-dirs]
```
Investigate the module for resolving CPATH/LD_LIBRARY_PATH from CEA/HPC modules.Version 0.4Patryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/31Resolve pip / pyproject.toml warning2023-01-27T14:51:20+01:00Patryk KiepasResolve pip / pyproject.toml warning```
DEPRECATION: mar-tools is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible repla...```
DEPRECATION: mar-tools is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
```Version 0.4Patryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/17Running the same MAR experiment can result in vastly different erros2023-02-09T17:48:12+01:00Patryk KiepasRunning the same MAR experiment can result in vastly different errosThese tests were made on spiritX machine, with mar-tools: [c08015609ff43804fbf78ba530995e4f3c1af887](https://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/commit/c08015609ff43804fbf78ba530995e4f3c1af887) and marv3-future: `64c757...These tests were made on spiritX machine, with mar-tools: [c08015609ff43804fbf78ba530995e4f3c1af887](https://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/commit/c08015609ff43804fbf78ba530995e4f3c1af887) and marv3-future: `64c75700aecc3d790e57a49f8a4a07f1686211aa` (`mariso_cross_compilation` branch).
# Example 1:
```
Call of PHYrad_CEP_mp OUT : 1/ 2/2015 23: 0: 0
OUTice x-hourly outputs 8 0.00000000
Writing of OUTice in ICE.20150201.ANx.nc: 2/ 2/2015 0: 0: 0
MAR time : 2/ 2/2015 0: 0: 0
Real time : 01/24/2023 11:37:20
Step time : dt=120.0, dtHyd= 60.0, dtDiff= 60.0, dtRadi= 3600. s, nt_Mix= 2
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
#0 0x14f2f4182d21 in ???
#1 0x14f2f4181ef5 in ???
#2 0x14f2f3e1b08f in ???
at /build/glibc-SzIz7B/glibc-2.31/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0
#3 0x55e74041bf50 in ???
#4 0x14f2f443c8e5 in ???
#5 0x55e74041cbee in ???
#6 0x55e74046c3a7 in ???
#7 0x55e7403b23ee in ???
#8 0x14f2f3dfc082 in __libc_start_main
at ../csu/libc-start.c:308
#9 0x55e7403b241d in ???
#10 0xffffffffffffffff in ???
srun: error: spiritx64-6: task 0: Segmentation fault (core dumped)
```
# Example 2:
```
ERROR filatmo 2015 2 1 0 6 for ( 4,123,19) -36.=> -9.
ERROR filatmo 2015 2 1 0 6 for ( 4,124,19) -46.=> -28.
STOP in filatmo.f: NaN on pixel(i,j,k) 4 126 19
-1299.78223 6.64605951 8.56959915
```
# Example 3:
```
Current / 1-Feb-2015 12: 2: 0 t = 2721720
2nd VBC / 1-Feb-2015 18 /(6) t = 2743200
CRASH1 in sisvat_qso.f on pixel (i,j,n) 25 96 1
decrease your time step or increase ntphys and ntdiff in time_steps.f
Note: The following floating-point exceptions are signalling: IEEE_INVALID_FLAG IEEE_DIVIDE_BY_ZERO IEEE_UNDERFLOW_FLAG IEEE_DENORMAL
```
On Spirit:
```
Current / 1-Feb-2015 12: 2: 0 t = 2721720
2nd VBC / 1-Feb-2015 18 /(6) t = 2743200
CRASH1 in sisvat_qso.f on pixel (i,j,n) 116 13 1
decrease your time step or increase ntphys and ntdiff in time_steps.f
Note: The following floating-point exceptions are signalling: IEEE_INVALID_FLAG IEEE_DIVIDE_BY_ZERO IEEE_OVERFLOW_FLAG IEEE_UNDERFLOW_FLAG IEEE_DENORMAL
```
# Example 4:
On spirit with intel (`spirit_intel` configuration)
```
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
mar 0000000000A83933 Unknown Unknown Unknown
libpthread-2.31.s 00001482CE15F420 Unknown Unknown Unknown
libiomp5.so 00001482CE2AA98D Unknown Unknown Unknown
libiomp5.so 00001482CE2A860E Unknown Unknown Unknown
libiomp5.so 00001482CE2A8224 Unknown Unknown Unknown
libiomp5.so 00001482CE29FE03 Unknown Unknown Unknown
libiomp5.so 00001482CE2A07BB Unknown Unknown Unknown
mar 0000000000AAA960 Unknown Unknown Unknown
mar 0000000000A6F735 Unknown Unknown Unknown
mar 00000000005E218D outice_ 190 outice.f90
mar 000000000041C10A MAIN__ 2460 mar.f90
mar 000000000040DB22 Unknown Unknown Unknown
libc-2.31.so 00001482CDE2C083 __libc_start_main Unknown Unknown
mar 000000000040DA2E Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
mar 0000000000A839D9 Unknown Unknown Unknown
libpthread-2.31.s 00001482CE15F420 Unknown Unknown Unknown
libiomp5.so 00001482CE2A5207 Unknown Unknown Unknown
libiomp5.so 00001482CE29DB0F Unknown Unknown Unknown
libiomp5.so 00001482CE29D89E Unknown Unknown Unknown
libiomp5.so 00001482CE22BDE1 Unknown Unknown Unknown
ld-2.31.so 00001482CEEA6F8D Unknown Unknown Unknown
libc-2.31.so 00001482CDE4E8A7 Unknown Unknown Unknown
libc-2.31.so 00001482CDE4EA60 on_exit Unknown Unknown
mar 0000000000A7F0E1 Unknown Unknown Unknown
mar 0000000000A83933 Unknown Unknown Unknown
libpthread-2.31.s 00001482CE15F420 Unknown Unknown Unknown
libiomp5.so 00001482CE2AA98D Unknown Unknown Unknown
libiomp5.so 00001482CE2A860E Unknown Unknown Unknown
libiomp5.so 00001482CE2A8224 Unknown Unknown Unknown
libiomp5.so 00001482CE29FE03 Unknown Unknown Unknown
libiomp5.so 00001482CE2A07BB Unknown Unknown Unknown
mar 0000000000AAA960 Unknown Unknown Unknown
mar 0000000000A6F735 Unknown Unknown Unknown
mar 00000000005E218D outice_ 190 outice.f90
mar 000000000041C10A MAIN__ 2460 mar.f90
mar 000000000040DB22 Unknown Unknown Unknown
libc-2.31.so 00001482CDE2C083 __libc_start_main Unknown Unknown
mar 000000000040DA2E Unknown Unknown Unknown
srun: error: spirit64-01: task 0: Exited with exit code 174
```Version 0.4Patryk KiepasPatryk Kiepashttps://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools/-/issues/12Improve documentation2023-01-30T15:04:36+01:00Patryk KiepasImprove documentationAdd better description of what happens when activating for the first/second time the env.
Write about creating new conda env from a file with pip: git+... dependency
```
name: MAR
channels:
- conda-forge
- defaults
dependencies:
-...Add better description of what happens when activating for the first/second time the env.
Write about creating new conda env from a file with pip: git+... dependency
```
name: MAR
channels:
- conda-forge
- defaults
dependencies:
- git
- fpm
- pip
- pip:
- fypp
- ford
- fprettify
- git+https://gitlab.in2p3.fr/ipsl/projets/awaca/models/mar-tools.git
```Version 0.4Patryk KiepasPatryk Kiepas