# Toolbox Guide

This folder contains post-processing, plotting, utility, and code-analysis scripts.

File headers/docstrings are the authoritative source for per-script behavior;
this page is a quick index and workflow-oriented entry point.

## Tools Summary

| File | Purpose |
|---|---|
| `analyse_netcdf.py` | Print min/max/mean and anomaly flags for numeric NetCDF variables |
| `concat_pem.py` | Concatenate `{basename}<index>.nc` files along `Time` |
| `display_netcdf.py` | Plot variables from NetCDF files (maps, sections, profiles) |
| `visu_layering.py` | Plot stratification diagnostics from one restart file |
| `visu_layering_evo.py` | Plot stratification evolution over many restart files |
| `modify_startfi_var.sh` | Set one variable value in `startfi.nc` |
| `modify_startfi_orbit.sh` | Set fixed orbital parameters in `startfi.nc` |
| `ini_pem_orbit.sh` | Initialize orbital parameters in `startfi.nc` from `run_pem.def` date and orbital forcing table |
| `multiple_exec.sh` | Run one script in multiple first-level subdirectories |
| `make_fortran_stats.sh` | Generate Fortran code statistics and CSV exports |
| `make_fortran_deps.sh` | Build Fortran dependency graph and compile-order analysis reports |
| `display_netcdf.yml` | Conda environment used only for `display_netcdf.py` |
| `MOLA_1px_per_deg.npy` | Reference topography grid used by map plotting workflows |
| `molaTeam_contour_31rgb_steps.csv` | Colormap/contour definition used by map plotting workflows |

## Prerequisites

Recommended Python environment for `display_netcdf.py` only:

```bash
cd PEM/toolbox
conda env create -f display_netcdf.yml
conda activate display_netcdf
```

Main Python dependencies: `netCDF4`, `xarray`, `numpy`, `matplotlib`, `cartopy`, `pandas`, `scipy`.

Optional: `vedo` for 3D display in `display_netcdf.py`.

System dependencies for shell helpers:

- `ncap2` (NCO)
- `bc`

Debian/Ubuntu example:

```bash
sudo apt-get install nco bc
```

## Practical Usage

### 1. Analyze one NetCDF file

```bash
python analyse_netcdf.py path/to/file.nc
```

Interactive mode:

```bash
python analyse_netcdf.py
```

### 2. Concatenate PEM outputs

```bash
python concat_pem.py --folder diags --basename diagevo --start 1 --end 10 --output merged.nc
```

Interactive mode:

```bash
python concat_pem.py
```

### 3. Plot diagnostics

```bash
python display_netcdf.py
```

Interactive mode is currently the most reliable path.

### 4. Layering diagnostics

```bash
python visu_layering.py
python visu_layering.py starts/restartevo1.nc
python visu_layering_evo.py
```

### 5. Update startfi.nc fields

```bash
./modify_startfi_var.sh
./modify_startfi_orbit.sh
```

Edit script parameters first.

### 6. Initialize orbit from run_pem.def

Required in working directory:

- `startfi.nc`
- `run_pem.def` with `pem_ini_earth_date=...`
- optional `orbitdata_path=...` in `run_pem.def`
- referenced orbital ASCII file (recommended `obl_ecc_lsp_mars.asc` from `PEM/datadir/`)

Example from simulation directory:

```bash
cp /path/to/PEM/datadir/obl_ecc_lsp_mars.asc .
./ini_pem_orbit.sh
```

### 7. Run one script in multiple subdirectories

Edit `multiple_exec.sh` (`script_name`, `tempdir_name`), then run:

```bash
./multiple_exec.sh
```

### 8. Generate code statistics

```bash
/path/to/PEM/toolbox/make_fortran_stats.sh
/path/to/PEM/toolbox/make_fortran_stats.sh --csv
```

### 9. Generate dependency analysis

```bash
/path/to/PEM/toolbox/make_fortran_deps.sh src
```

Common filter for large trees:

```bash
/path/to/PEM/toolbox/make_fortran_deps.sh --prune top:40 --long-range-threshold 2 src
```

## Dependency Outputs

Generated in current working directory:

- `fortran_deps.svg`, `fortran_deps.png`: dependency graph visualization
- `fortran_deps.dot`: Graphviz source
- `cycle_report.txt`: circular dependency groups
- `compile_layers.txt`: topological compile layers
- `critical_path.txt`: longest dependency chain
- `module_centrality.txt`: fan-in ranking
- `deps.mk`: makefile dependency snippet

How to read the graph:

- warm colors: higher fan-in modules
- dashed edges: long-range dependencies across layers
- red path/edges: critical compile path

Doxygen integration:

- `Code Metrics` top-menu tab embeds graph and exposes all artifacts.

## Data Conventions

Common coordinates/dimensions:

- time: `Time`, `time`, or `time_counter`
- latitude: `latitude` or `lat`
- longitude: `longitude` or `lon`
- slopes: `nslope`
- stratification: `nb_str_max`

Map assets available in this folder for `display_netcdf.py`:

- `MOLA_1px_per_deg.npy`
- `molaTeam_contour_31rgb_steps.csv`

## Known Limitations

- `display_netcdf.py`: CLI and plotting internals are not fully aligned; prefer interactive usage
- `display_netcdf.py`: `--show-polar` and `--show-3d` partly rely on interactive prompts
- `concat_pem.py`: expects a `Time` coordinate and applies cumulative offset
- `analyse_netcdf.py`: reads full variable arrays into memory
- `visu_layering_evo.py`: orbital ASCII prompt behavior differs from restart defaults
- `multiple_exec.sh`: scans one subdirectory level only

## Contributor Notes

- Use `make_fortran_deps.sh` and `make_fortran_stats.sh` before large refactors to detect coupling growth and heavy modules.
- Prioritize high fan-in modules for API clarity, comments, and regression checks.

Practical mapping to code-quality decisions:

- maintainability/discoverability: use fan-in ranking to identify modules that need clearer interfaces and comments,
- reliability/robustness: use compile layers and critical path outputs to stage risky refactors,
- efficiency (memory/compute): use file/directory stats to spot unusually large or complex hotspots before optimization,
- extensibility: keep new feature modules off critical hubs unless reuse is intended.

## Related Docs

- `../USER_GUIDE.md`: build, run, coding guidance, and version/provenance commands
- `../datadir/README.md`: runtime data files
- `../deftank/README.md`: chained workflow setup
- `../doc/README.md`: documentation index and Doxygen usage