-------------------
Added
^^^^^
* `senwu`_: Support fp16 optimization.
(`77 <https://github.com/SenWu/emmental/pull/77>`_)
* `senwu`_: Support distributed learning.
(`78 <https://github.com/SenWu/emmental/pull/78>`_)
* `senwu`_: Support no label dataset.
(`79 <https://github.com/SenWu/emmental/pull/79>`_)
* `senwu`_: Support output model immediate_ouput.
(`80 <https://github.com/SenWu/emmental/pull/80>`_)
.. note::
To output model immediate_ouput, the user needs to specify which module output
he/she wants to output in `EmmentalTask`'s `action_outputs`. It should be a pair of
task_flow name and index or list of that pair. During the prediction phrase, the
user needs to set `return_action_outputs=True` to get the outputs where the key is
`{task_flow name}_{index}`.
.. code:: python
task_name = "Task1"
EmmentalTask(
name=task_name,
module_pool=nn.ModuleDict(
{
"input_module": nn.Linear(2, 8),
f"{task_name}_pred_head": nn.Linear(8, 2),
}
),
task_flow=[
{
"name": "input",
"module": "input_module",
"inputs": [("_input_", "data")],
},
{
"name": f"{task_name}_pred_head",
"module": f"{task_name}_pred_head",
"inputs": [("input", 0)],
},
],
loss_func=partial(ce_loss, task_name),
output_func=partial(output, task_name),
action_outputs=[
(f"{task_name}_pred_head", 0),
("_input_", "data"),
(f"{task_name}_pred_head", 0),
],
scorer=Scorer(metrics=task_metrics[task_name]),
)
* `senwu`_: Support action output dict.
(`82 <https://github.com/SenWu/emmental/pull/82>`_)
* `senwu`_: Add a new argument `online_eval`. If `online_eval` is off, then model won't
return `probs`.
(`89 <https://github.com/SenWu/emmental/pull/89>`_)
* `senwu`_: Support multiple device training and inference.
(`91 <https://github.com/SenWu/emmental/pull/91>`_)
.. note::
To train model on multiple devices such as CPU and GPU, the user needs to specify
which module is on which device in `EmmentalTask`'s `module_device`. It's a
ditctionary with key as the module_name and value as device number. During the
training and inference phrase, the `Emmental` will automatically perform forward
pass based on module device information.
.. code:: python
task_name = "Task1"
EmmentalTask(
name=task_name,
module_pool=nn.ModuleDict(
{
"input_module": nn.Linear(2, 8),
f"{task_name}_pred_head": nn.Linear(8, 2),
}
),
task_flow=[
{
"name": "input",
"module": "input_module",
"inputs": [("_input_", "data")],
},
{
"name": f"{task_name}_pred_head",
"module": f"{task_name}_pred_head",
"inputs": [("input", 0)],
},
],
loss_func=partial(ce_loss, task_name),
output_func=partial(output, task_name),
action_outputs=[
(f"{task_name}_pred_head", 0),
("_input_", "data"),
(f"{task_name}_pred_head", 0),
],
module_device={"input_module": -1, f"{task_name}_pred_head": 0},
scorer=Scorer(metrics=task_metrics[task_name]),
)
* `senwu`_: Add require_prob_for_eval and require_pred_for_eval to optimize score
function performance.
(`92 <https://github.com/SenWu/emmental/pull/92>`_)
.. note::
The current approach during score the model will store probs and preds which might
require a lot of memory resources especially for large datasets. The score function
is also used in training. To optimize the score function performance, this PR
introduces two new arguments in `EmmentalTask`: `require_prob_for_eval` and
`require_pred_for_eval` which automatically selects whether `return_probs` or
`return_preds`.
.. code:: python
task_name = "Task1"
EmmentalTask(
name=task_name,
module_pool=nn.ModuleDict(
{
"input_module": nn.Linear(2, 8),
f"{task_name}_pred_head": nn.Linear(8, 2),
}
),
task_flow=[
{
"name": "input",
"module": "input_module",
"inputs": [("_input_", "data")],
},
{
"name": f"{task_name}_pred_head",
"module": f"{task_name}_pred_head",
"inputs": [("input", 0)],
},
],
loss_func=partial(ce_loss, task_name),
output_func=partial(output, task_name),
action_outputs=[
(f"{task_name}_pred_head", 0),
("_input_", "data"),
(f"{task_name}_pred_head", 0),
],
module_device={"input_module": -1, f"{task_name}_pred_head": 0},
require_prob_for_eval=True,
require_pred_for_eval=True,
scorer=Scorer(metrics=task_metrics[task_name]),
)
* `senwu`_: Support save and load optimizer and lr_scheduler checkpoints.
(`93 <https://github.com/SenWu/emmental/pull/93>`_)
* `senwu`_: Support step based learning and add argument `start_step` and `n_steps` to
set starting step and total step size.
(`93 <https://github.com/SenWu/emmental/pull/93>`_)
Fixed
^^^^^
* `senwu`_: Fix customized optimizer support issue.
(`81 <https://github.com/SenWu/emmental/pull/81>`_)
* `senwu`_: Fix loss logging didn't count task weight.
(`93 <https://github.com/SenWu/emmental/pull/93>`_)