Ripple1d

This PR step tunes a hyperparameter in the conflation process that was causing large HEC-RAS source models to not conflate with any NWM reaches.

A central step of model conflation is identifying the relevant NWM reaches within the model domain. To do this, a valid path of NWM reaches is found that stretches from the upstream end of the HEC-RAS model to the downstream end. The valid path is found by walking several potential paths and finding one that connects upstream to downstream. A max walk length was included in earlier ripple versions to prevent process deadlocking, however, the limit was set to 100 reaches. This limit was too low for many large HEC-RAS models.

Based on a performance analysis, it was found that setting a large number (3,000) for the iteration limit would not hurt conflation speed and would allow conflation of a HEC-RAS model the full length of the Missouri River.


This release resolves a minor bug that improves conflation at confluences and a rather major bug that was causing truncation of the profile names when the profile names exceeded 16 characters.

To resolve the truncation of profile names exceeding 16 characters, each profile name is now designated an integer value (0,1,2,3, etc.) to be used as the "new" profile name in the HEC-RAS flow file. A mapping of old-to-new profile name was then stored in the description of the flow file as a json string. During post-processing this map is retrieved and used to re-map the integer profile names back to the previous naming convention for profiles.

At some confluences, a NWM reach may intersect cross-sections from both tributary HEC-RAS reaches. To determine the "correct" reach for conflation, Ripple1D selects the reach with more coverage length. At times, the reach with more coverage would lead to the NWM reach being conflated in reverse. This release handles this situation by removing any cross-sections that would lead to reverse conflation from the set of potential HEC-RAS reaches.

If this text is confusing, see the new conflation test N.


This release addresses several bugs that were identified in v0.10.0.


Bugfixes Addressed

Ignore terrain agreement errors
A new argument `terrain_agreement_ignore_error` was added to the `create_ras_terrain` endpoint. It's default value is false, which will maintain the current functionality. When set to true, if an error is raised in the terrain agreement calculation step, the error will be logged and no terrain agreement database will be created. The endpoint will still respond with a `"terrain_agreement"` key, but it will have an empty value.

Handle river_reach names more generically
While improving the conflation logic in 0.10.0, we made an assumption that the river_reach identifier in a geometry file would follow the default HEC-RAS formatting. Some contractors do not follow this formatting, so we have updated the code to more generically store river_reach names instead of forcing the default RAS formatting.

Add Model Name to extract_submodel
A required arg, `model_name` has been added to the `extract_submodel` endpoint. This enables the the user to specify the name of the model instead of inferring it from the model directory name which was not full proof.


closes 326
closes 325
closes 327

This PR addresses several issues relating to the conflation process. Those issues are grouped in [the 0.10.0 milestone](https://github.com/Dewberry/ripple1d/milestone/13), and this PR successfully addresses all of them and more. Specific changes are detailed in the subsections below.


Features Added

Check Units
The source HEC-RAS model (and thus the derived submodel) must have units of `English`. If it does not then an error is raised in the `create_model_run_normal_depth` endpoint. To do this several additional changes had to be made to the schema.
- `units` is now passed from the source_model.gpkg to the conflation.json during `conflate_model`.
- `source_model_metadata` is now passed from the conflation.json to the ripple1d.json during `extract_submodel`.
- `units` and `steady` (which is a boolean indicating if the source model was steady state) is now passed from the conflation.json to the submodel.gpkg during `extract_submodel`. This required adding a new metadata table to the submodel.gpkg.
- `create_model_run_normal_depth` reads the ripple1d.json and raises an error if `units` is not `English`.

Handle Unsteady Source Model
The handling of unsteady flow files in the source RAS model has been updated. Unsteady RAS models can now be extracted for cataloging purposes without error. However, it was decided that unsteady 1d models should not be processed through ripple1d as a steady state model and so an error is raised during `create_model_run_normal_depth` if the `forcing file extension` from the source model does not contains `f` indicating that it is not a steady state model.

Identify Overtopped Cross Sections
A column named `xs_overtopped` has been to the rating curve database which indicates if any cross sections were overtopped for any plan/profile. This should help to identify when a FIM should be scrutinized, discarded, or other decision made with regards to how it should be interpret it.

Add Model Name as arg
A required arg, `model_name` has been added to the `conflate_model` and `compute_conflation_metrics` endpoint. This enables the the user to specify the name of the model instead of inferring it from the model directory name which was not full proof.

Confluence Improvements

Issue 222 identified errors in conflation in confluence areas. The conflation behavior has been modified to enforce that both tributary sub models will share a downstream cross-section with the upstream cross-section of the downstream sub model.

Eclipsed Reach Identification

An eclipsed reach is defined as a NWM reach that has upstream and downstream NWM reaches conflated but with no source model cross-sections intersecting itself. In previous versions of the code, ripple was both under- and over-identifying eclipsed reaches (see 311 and 316). The code has been improved to properly scan for eclipsed NWM reaches.

Conflating NWM Reaches With Low Source Model Overlap

Issues 218 and 227 detail edge cases near first-order NWM reaches. The conflation process has been updated to allow for a) scanning for NWM that intersect a small portion of the source model and b) expanding the sub model upstream cross-section above the NWM terminus if an additional cross-section is available.

Documentation and testing

13 new tests were added to prevent code regression as the conflation module was updated. These test cases also serve as examples of how conflation "should" behave. As such, graphics were generated for each test and have been added to the [conflate_model endpoint page](https://ripple1d.readthedocs.io/en/stable/endpoints/conflate_model.html).



closes 186
closes 303
closes 123
closes 176
closes 222
closes 311
closes 227
closes 218
closes 316

Summary
This release of `ripple1d` fixes several bugs that were identified in a large production run. Those bugs were detailed and summarized in 268.

Bug Fixes
- When a new terrain is downloaded but contains only nodata values, `NullTerrainError` is raised, and the endpoint process will fail.
- When a new terrain is downloaded but the HEC-RAS CreateTerrain subprocess fails silently, the output is checked, and a `RasTerrainFailure` error is raised. Additionally, bad_projection_test.py has been added to test that the error will raise.
- When a source model is conflated such that the most downstream XS for a NWM reach a has higher river station than the most upstream reach, a `BadConflation` error is raised.
- When sub model extraction is attempted on a reach that would have one or zero cross-sections but is not tagged as eclipsed, a `SingleXSModel` error is raised. The endpoint response was also cleaned up such that eclipsed reaches no longer return a path to a nonexistent 123.ripple1d.json. Logging has been improved for all these cases.
- flow indexing in the create_flow_depth_combinations function has been improved to avoid rounding values. A test has been added for one of the models that was causing problems.
- Handling of interpolated cross-sections has been improved such that they may have sub models created, may be run, and may be used in the create FIM lib steps. A new test model -- winooski -- has been added with interpolated cross-sections.
- Null (blank) downstream reach values are now loaded as 0. This is consistent with how HEC-RAS handles them. The new winooski model contains null downstream reach length at the most downstream cross-section.

Closes 281
Closes 282
Closes 291
Closes 292
Closes 295
Closes 298

> [!IMPORTANT]
> Version 0.8.0 was released without having incorporated bugfixes added after v0.8rc1. This update adds in the necessary bugfixes to reflect the most up-to-date version of the software. The text below is a copy of the 0.8.0 release notes.

Summary

This release adds a suite of diagnostic metrics on model quality and terrain agreement to the ripple1d outputs. Model quality metrics reflect how well a HEC-RAS model conforms to standards of practice for developing HEC-RAS models and provide easy access to many model attributes that would be checked during an engineering model QC. Terrain agreement metrics quantify how well the terrain generated in the create_ras_terrain endpoint matches the source model terrain. Access to these metrics give ripple1d users actionable information to inform which HEC-RAS models should be preferred when models overlap. Furthermore, elevation-specific terrain agreement metrics can be used to alert users when a FIM product substantially disagrees with a source model geometry (indicative of lower FIM quality).

Additional features have been added to handle HEC-RAS warnings about hydraulic table start points and cross-section naming.

Features Added

Hydraulic Table Start Elevation

When a hydraulic table for a cross-section has a minimum elevation below the section invert, HEC-RAS generates a pop up warning as demonstrated in https://github.com/Dewberry/ripple1d/issues/249. In the warning, the software specifies that any subsequent saves to the geometry file will automatically adjust the hydraulic table minimum elevation to be above the section invert. To mimic this functionality, a new function call has been added to the RasGeomText constructor. The call will check for any occurrences of the htab issue, adjust the minimum elevation, and overwrite the geometry file with the updated value.

HEC-RAS implements some convoluted process to update both the htab start value and the increments. We elected to simply update the htab start elevation to the cross-section invert and preserve the original engineer-determined increments.

Sub Model Cross-Section Names

In earlier versions of ripple1d, the software attempted to carry cross-section names from source model to sub model. When that naming would have resulted in decreasing values in the upstream direction, the stationing was adjusted to preserve increasing values in the names. The renaming process led to a couple of bugs in downstream processes (ex 225 and 266). Since cross-section names do not affect hydraulic computation, the software will now rename all sub model cross-sections to an autoincremented value (ex 1, 2, 3, etc).

Additionally, the extract_submodel endpoint has been refactored to fix a potential bug in the junction walking step. For cases where more than one junction separated the u/s and d/s cross-sections or cases where the junctions were not reported from upstream to downstream in the HEC-RAS geometry file, the extract_submodel endpoint would only grab a single intermediate reach. No examples of this behavior have been found in production, but the bug was tested in development with a testing HEC-RAS model.

Terrain Agreement Metrics

A new step has been added to the create_ras_terrain endpoint that will generate a json of terrain agreement metrics. The json is saved to a file with suffix ".terrain_agreement.json" that is saved to the sub model directory. The JSON contains a suite of metrics that have been documented in the Technical Summary of the readthedocs page (https://ripple1d.readthedocs.io/en/latest/tech_summary.html) and the create_ras_terrain endpoint page (https://ripple1d.readthedocs.io/en/latest/endpoints/create_ras_terrain.html).

Model Quality Metrics

A new step has been added to the extract source model endpoint which extracts source model information to determine the quality of the source model. These metrics are populated in the cross section layer located in the source model geopackage layer for every cross section. The submodel will inherit these metrics during the subset gpkg phase.

Bug Fixes
- A feature was added in v0.7.0 to allow html rendering of the jobs endpoint, however, the endpoint was not updated to accept a flag toggling between json and html responses. Additionally, an oversight in the project manifest meant that the html and css templates for the jobs page were not transferred to ripple1d when installed via pip. This has been resolved so that the jobs endpoint returns a json response by default, and an html response will be returned when the key-value pair f=html is in the query string.
- readthedocs was failing to build our documentation on their server. The docs folder has been updated such that it will build via their service. Additionally, the changelog will autogenerate now. Some old windows dependencies and imports have been removed since they are not used to run HEC-RAS any more and needed unnecessary logic to avoid breaking the readthedocs build service.

Closes 118
Closes 207
Closes 208
Closes 209
Closes 225
Closes 249
Closes 266
Closes 267
Closes 268
Closes 269
Closes 270