Highlights of the v0.7.2 release 🚀🚀
In this version, we prioritize issues that were brought to our attention by our users. More specifically, we addressed various issues related with usability and performance, and we improved the way we inform users about how the API itself works as they run their simulations.
Some of the main improvements include:
- More explicit information about errors or problems encountered during the simulations, such as, alerting the user that the reasons why the simulation failed was due to lack of disk space. In this version, there are a lot more potential errors / problems being reported back to the user because we improved the way we communicate errors from the backend to the client ([pr886](https://github.com/inductiva/inductiva-web-api/pull/886)). This dramatically improves usability because it gives actional clues on how to correct problems.
- A mechanism for grouping tasks under a single addressable concept called [Projects](https://tutorials.inductiva.ai/intro_to_api/projects.html). This is especially useful when the user needs to run several simulations under a single context (“a project”), such as when exploring variations of a certain simulation use-case, or when needing to generate datasets of synthetic data.
- We are also now making available ways for users to get summary information about their tasks. The new print_summary() method provides users with a number of stats related to the different stages of the simulations, as well as to the data produced.
- The user now has more direct information about quotas and pricing. This helps users better manage the amount of resources they can use, and also understand how they can save costs by properly choosing specific configurations.
List of changes
[188](https://github.com/inductiva/tasks/issues/188)
Improved messaging: whenever the user tries to access a task that they don’t own or that doesn’t exist, namely when the task ID was misspelled, the user is alerted of the issue by a message automatically printed on the CLI.
[199](https://github.com/inductiva/tasks/issues/199)
Standard streams (stderr for errors and stdout for output) are automatically downloaded during the execution of task.wait(), so that they can be accessed even if the task is not completed.
Although task.download_outputs() is maintained as an available method to download the output of a finished task, this extra measure guarantees that users will have access to errors and output even in the cases where something goes wrong with the task execution.
A message is printed to inform the user when the download is finished.
The command line logs provide the download location.
[200](https://github.com/inductiva/tasks/issues/200)
New task.get_output_info() that can be called by the user to retrieve information about the task’s output, even before downloading it.
This method returns one object with four attributes:
- output_info.total_size_bytes - total output size, before compression;
- output_info.total_compressed_size_bytes - total output size, after compression;
- output_info.n_files - total number of files in the output;
- output_info.files - same attributes per file in the output.
[205](https://github.com/inductiva/tasks/issues/205) [#206](https://github.com/inductiva/tasks/issues/206)
To [terminate all](https://docs.inductiva.ai/en/latest/how_to/manage_computational_resources.html#terminate-the-active-computational-resources) active computational resources becomes much faster.
After the user confirms the instruction, it’ll take only a few seconds to terminate all their active resources, avoiding idle resources and guaranteeing an effective user quota management.
A message is printed as soon as the instruction is finished.
[203](https://github.com/inductiva/tasks/issues/203)
When the user submits a task, information is provided about the number of tasks ahead of theirs. This information is updated regularly.
Task sokraaaw70txt5jc65it8h7kw successfully queued and waiting to be picked-up for execution...
Number of tasks ahead of task sokraaaw70txt5jc65it8h7kw in queue: 17
[196](https://github.com/inductiva/tasks/issues/196)
Manage remote storage - options to delete the data remotely, after download:
- New method task.remove_remote_files() that removes the output.
- New optional parameter ‘rm_remote_files’ in task.download_outputs(). Default is False, so the parameter must be explicitly set to True when the user wants to delete the data after downloading.
[192](https://github.com/inductiva/tasks/issues/192), [#198](https://github.com/inductiva/tasks/issues/198), [#197](https://github.com/inductiva/tasks/issues/197)
New method task.print_summary(), that shows the times spent at all stages of the process, including all auxiliary tasks, such as moving data around between your local computer, your personal remote storage space and the executer machine.
Task status: success
Wall clock time: 14.05 s
Time breakdown:
Input upload: 0.38 s
Time in queue: 0.00 s
Container image download: 1.08 s
Input download: 0.06 s
Input decompression: 0.02 s
Computation: 12.16 s
Output compression: 0.10 s
Output upload: 0.14 s
Data:
Size of zipped output: 878.53 KB
Size of unzipped output: 2.05 MB
Number of output files: 33
Check out an [example](https://docs.staging.inductiva.ai/simulators/XBeach.html#a-more-advanced-example) of how to take advantage of this method in our tutorials documentation.
Here’s the cheat sheet:
- Wall clock time: sum of all times;
- Input upload: time to upload the input files;
- Time in queue: time waiting for task to be picked up;
- Input download: time to download from the remote storage (bucket) to the machine where the task was run;
- Computation: time to actually run the task;
- Output upload: time to upload from the machine where the task was run to the remote storage.
[193](https://github.com/inductiva/tasks/issues/193)
Information about the available [user quota](https://docs.inductiva.ai/en/latest/api_reference/user_quotas.html#) is displayed more often:
- ‘[inductiva resources list](https://docs.inductiva.ai/en/latest/cli/managing-resources.html#list-active-resources)’ overview now also provides the maximum cost ($/hour) of each of the active computational resources.
- When creating or starting a resource,
-- a summary of the quotas being used is shown in a table format;
-- a savings suggestion is provided, when applicable (look for the ‘>>’ after the estimated cloud cost).
Registering MachineGroup configurations:
> Name: api-efmv0es2vnwkyokm5ne1t2nmy
> Machine Type: c2-standard-4
> Data disk size: 10 GB
> Number of machines: 1
> Spot: False
> Estimated cloud cost of machine group: 0.230 $/h
>> The same machine group with spot instances would cost 0.182 $/h less per machine (79.43% savings). Specify `spot=True` in the constructor to use spot machines.
Starting MachineGroup(name="api-efmv0es2vnwkyokm5ne1t2nmy"). This may take a few minutes.
Note that stopping this local process will not interrupt the creation of the machine group. Please wait...
Machine Group api-efmv0es2vnwkyokm5ne1t2nmy with c2-standard-4 machines successfully started in 0:00:24.
The machine group is using the following quotas:
NAME USED BY RESOURCE NEW TOTAL USAGE MAX ALLOWED
cost_per_hour 0.22968 0.22968 270
total_num_machines 1 1 100
total_num_vcpus 4 4 1500
- When terminating the resource, a summary of the quotas being released is shown.
Successfully requested termination of MachineGroup(name="api-efmv0es2vnwkyokm5ne1t2nmy").
Termination of the machine group freed the following quotas:
NAME FREED BY RESOURCE NEW TOTAL USAGE MAX ALLOWED
cost_per_hour 0.22968 0 270
total_num_machines 1 0 100
total_num_vcpus 4 0 1500
[194](https://github.com/inductiva/tasks/issues/194), [#195](https://github.com/inductiva/tasks/issues/195)
Additional optional sub-methods to access logs.
We created a bunch of sub-methods to easy access and get the most out of the logs being generated.
| Instruction | Print to CLI the logs |
| :----- | :------ |
| inductiva logs task_id | of the task identified by task_id |
| inductiva logs | of the last submitted task |
| inductiva logs submitted | of the last submitted task |
| inductiva logs submitted-1 | of the 2nd to last submitted task |
| inductiva logs submitted-n | of the last (n+1)th submitted task |
| inductiva logs started | of the last started task |
| inductiva logs task_id stdout | Print to CLI only the outputsof the task identified by task_id |
| inductiva logs task_id stderr | Print to CLI only the errorsof the task identified by task_id |
| inductiva logs task_id stdout stderr | Print to CLI the outputs and errorsof the task identified by task_id. <br> By default, errors will be printed in red to be distinguishable from other prints. Use no-color to disable the colorized output.
| Combinations of stdout / stderrwith submitted / started | Print to CLI the outputs and / or errorsof the task identified by instruction, as described above. |