Oneflow

Highlights

- First class support for eager execution. The deprecated APIs are moved to `oneflow.compatible.single_client`
- Drop-in replacement of `import torch` for existing Pytorch projects. You could test it by inter-changing `import oneflow as torch` and `import torch as flow`.
- [nn.Module](https://docs.oneflow.org/master/basics/04_build_network.html#module) for eager execution
- [nn.Graph](https://docs.oneflow.org/master/basics/08_nn_graph.html) for lazy execution
- [DDP](https://oneflow.readthedocs.io/en/master/nn.html#oneflow.nn.parallel.DistributedDataParallel) for data parallel

A sneak peek of the new API

Here is a minimum example showcasing how to incorporate a `nn.Module` in a `nn.Graph` and have it run in lazy mode.

python
class NeuralGraph(flow.nn.Graph):
def __init__(self, ...):
super().__init__()
self.model = model model is a nn.Module instance

def build(self, x):
y_pred = self.model(x)
return y_pred

graph = NeuralGraph() to create a nn.Graph instance
y_pred = graph(x) to run the created nn.Graph


New in Python API

- [**feature**][**eager**][**op**][**test**][**python**][**interface**] Add test for convtranspose2d [5239](https://github.com/Oneflow-Inc/oneflow/pull/5239)
- [**enhancement**][**python**][**interface**] Add GroupNorm [5175](https://github.com/Oneflow-Inc/oneflow/pull/5175)
- [**enhancement**][**eager**][**python**][**interface**] [Add] avgpool1d avgpool3d [5165](https://github.com/Oneflow-Inc/oneflow/pull/5165)
- [**feature**][**eager**][**op**][**python**][**interface**] Add deconv cpu impl [5224](https://github.com/Oneflow-Inc/oneflow/pull/5224)
- [**bug**][**eager**][**api**][**python**][**interface**] Fix acosh bug [5221](https://github.com/Oneflow-Inc/oneflow/pull/5221)
- [**feature**][**eager**][**op**][**python**][**interface**] Dev modules ctc loss [5168](https://github.com/Oneflow-Inc/oneflow/pull/5168)
- [**bottleneck**][**bug**][**documentation**][**python**][**interface**] Fix meshgrid test bug [5208](https://github.com/Oneflow-Inc/oneflow/pull/5208)
- [**eager**][**documentation**][**python**][**interface**] Rename CosineScheduler to CosineAnnealingLR [5112](https://github.com/Oneflow-Inc/oneflow/pull/5112)
- [**feature**][**eager**][**python**][**interface**] Add meshgrid module [5205](https://github.com/Oneflow-Inc/oneflow/pull/5205)
- [**enhancement**][**feature**][**bug**][**op**][**python**] support bias in conv2d's parameter list [5322](https://github.com/Oneflow-Inc/oneflow/pull/5322)
- [**eager**][**documentation**][**api**][**python**][**interface**] add not_equal, greater_equal and less_equal module [5350](https://github.com/Oneflow-Inc/oneflow/pull/5350)
- [**enhancement**][**eager**][**python**] refine pow module and its test [5319](https://github.com/Oneflow-Inc/oneflow/pull/5319)
- [**enhancement**][**eager**][**op**][**python**] Add triu op [5329](https://github.com/Oneflow-Inc/oneflow/pull/5329)
- [**enhancement**][**bug**][**python**] Fix optimizer for not supporting all kinds of iterables [5355](https://github.com/Oneflow-Inc/oneflow/pull/5355)
- [**bug**][**python**][**interface**] raise IndexError in get_canonical_index to support for loop [5345](https://github.com/Oneflow-Inc/oneflow/pull/5345)
- [**bug**][**python**][**interface**] tensor slice assign supports broadcasting [5344](https://github.com/Oneflow-Inc/oneflow/pull/5344)
- [**enhancement**][**op**][**python**] add cpu group conv logic [5314](https://github.com/Oneflow-Inc/oneflow/pull/5314)
- [**enhancement**][**python**] Add 'nn.Mish' module and corresponding functions [5310](https://github.com/Oneflow-Inc/oneflow/pull/5310)
- [**enhancement**][**build**][**python**] Remove ONNX from setup py [5297](https://github.com/Oneflow-Inc/oneflow/pull/5297)
- [**enhancement**][**python**][**interface**] [add] zeropad2d [5278](https://github.com/Oneflow-Inc/oneflow/pull/5278)
- [**feature**][**system**][**python**][**interface**] Lazy nn.Graph FeedInputOpExpr [5458](https://github.com/Oneflow-Inc/oneflow/pull/5458)
- [**feature**][**python**][**interface**] integrate nn.image.flip [5411](https://github.com/Oneflow-Inc/oneflow/pull/5411)
- [**bug**][**python**] Fix issues in point of MultiClientSession [5469](https://github.com/Oneflow-Inc/oneflow/pull/5469)
- [**enhancement**][**bug**][**python**] update HasAllMultiClientEnvVars() [5459](https://github.com/Oneflow-Inc/oneflow/pull/5459)
- [**enhancement**][**python**] Add in_top_k function [5428](https://github.com/Oneflow-Inc/oneflow/pull/5428)
- [**enhancement**][**python**] Dev add docstring [5449](https://github.com/Oneflow-Inc/oneflow/pull/5449)
- [**feature**][**api**][**python**] MultiClientSession [5407](https://github.com/Oneflow-Inc/oneflow/pull/5407)
- [**documentation**][**python**] remove --user [5431](https://github.com/Oneflow-Inc/oneflow/pull/5431)
- [**feature**][**python**][**interface**] nn.Graph python [5309](https://github.com/Oneflow-Inc/oneflow/pull/5309)
- [**feature**][**python**][**interface**] Fea/nn graph/graph name [5413](https://github.com/Oneflow-Inc/oneflow/pull/5413)
- [**bug**][**python**][**interface**] rm nn.Graph.train [5424](https://github.com/Oneflow-Inc/oneflow/pull/5424)
- [**op**][**documentation**][**api**][**python**][**interface**] add bernoulli module [5353](https://github.com/Oneflow-Inc/oneflow/pull/5353)
- [**enhancement**][**python**] flow.S/B/P [5306](https://github.com/Oneflow-Inc/oneflow/pull/5306)
- [**enhancement**][**documentation**][**python**] Add instruction on upgrade pip [5400](https://github.com/Oneflow-Inc/oneflow/pull/5400)
- [**enhancement**][**python**] Rm oneflow export and experimental [5589](https://github.com/Oneflow-Inc/oneflow/pull/5589)
- [**bug**][**python**] Fix nn.graph.utils module conflict [5598](https://github.com/Oneflow-Inc/oneflow/pull/5598)
- [**feature**][**ci**][**python**] Update autotest framework [5520](https://github.com/Oneflow-Inc/oneflow/pull/5520)
- [**enhancement**][**python**] copy of_proto_python_dir to compatible_single_client_python [5539](https://github.com/Oneflow-Inc/oneflow/pull/5539)
- [**enhancement**][**api**][**python**] del default env init [5537](https://github.com/Oneflow-Inc/oneflow/pull/5537)
- [**enhancement**][**python**] Fix single client using same glog file [5535](https://github.com/Oneflow-Inc/oneflow/pull/5535)
- [**bug**][**api**][**python**] Fix Session TryClose [5531](https://github.com/Oneflow-Inc/oneflow/pull/5531)
- [**enhancement**][**feature**][**python**] split vector-matrix norm [5478](https://github.com/Oneflow-Inc/oneflow/pull/5478)
- [**feature**][**eager**][**op**][**python**][**interface**] Add more upsample kernel [5382](https://github.com/Oneflow-Inc/oneflow/pull/5382)
- [**enhancement**][**feature**][**test**][**python**] add torchstyle unittest [5489](https://github.com/Oneflow-Inc/oneflow/pull/5489)
- [**feature**][**system**][**python**] nn.Graph with training [5662](https://github.com/Oneflow-Inc/oneflow/pull/5662)
- [**enhancement**][**feature**][**python**] Fea/nn graph/block proxy func [5727](https://github.com/Oneflow-Inc/oneflow/pull/5727)
- [**enhancement**][**api**][**python**] consistent_tensor_to_api [5703](https://github.com/Oneflow-Inc/oneflow/pull/5703)
- [**feature**][**eager**][**op**][**python**] Dev Align torch avgpool [5610](https://github.com/Oneflow-Inc/oneflow/pull/5610)
- [**enhancement**][**python**] fix circular deps of sbp python module [5706](https://github.com/Oneflow-Inc/oneflow/pull/5706)
- [**documentation**][**python**] [part5]Remove singleclient outdated api [5674](https://github.com/Oneflow-Inc/oneflow/pull/5674)
- [**enhancement**][**python**] [part4]Remove singleclient outdated api [5672](https://github.com/Oneflow-Inc/oneflow/pull/5672)
- [**bug**][**op**][**python**] remove outdated code in conv3d [5696](https://github.com/Oneflow-Inc/oneflow/pull/5696)
- [**enhancement**][**test**][**python**] enlarge tolerance of dataloader test [5689](https://github.com/Oneflow-Inc/oneflow/pull/5689)
- [**enhancement**][**test**][**python**] add autotest for some math ops [5646](https://github.com/Oneflow-Inc/oneflow/pull/5646)
- [**feature**][**python**] nn.Graph optimizer part 2: add L2, pass job complete, refactor [5604](https://github.com/Oneflow-Inc/oneflow/pull/5604)
- [**enhancement**][**python**] Add clip_grad_norm [5299](https://github.com/Oneflow-Inc/oneflow/pull/5299)
- [**purge**][**python**] Remove Single-Client API in oneflow default python [5827](https://github.com/Oneflow-Inc/oneflow/pull/5827)
- [**bug**][**python**] Fix ddp grad size [5834](https://github.com/Oneflow-Inc/oneflow/pull/5834)
- [**enhancement**][**feature**][**python**] Dev RMSprop graph conf [5768](https://github.com/Oneflow-Inc/oneflow/pull/5768)
- [**enhancement**][**purge**][**eager**][**python**] remove scale arg in optimizer [5821](https://github.com/Oneflow-Inc/oneflow/pull/5821)
- [**enhancement**][**feature**][**python**] graph/block io check [5803](https://github.com/Oneflow-Inc/oneflow/pull/5803)
- [**enhancement**][**feature**][**python**] Dev adam graph conf [5709](https://github.com/Oneflow-Inc/oneflow/pull/5709)
- [**purge**][**python**] [part10]Remove singleclient outdated api [5756](https://github.com/Oneflow-Inc/oneflow/pull/5756)
- [**feature**][**api**][**python**] better repr of nn.Graph for debug [5762](https://github.com/Oneflow-Inc/oneflow/pull/5762)
- [**bug**][**python**] fix weight decay in RMSprop [5755](https://github.com/Oneflow-Inc/oneflow/pull/5755)
- [**purge**][**python**] [part9]Remove singleclient outdated api [5752](https://github.com/Oneflow-Inc/oneflow/pull/5752)
- [**purge**][**python**] [part8]Remove singleclient outdated api [5750](https://github.com/Oneflow-Inc/oneflow/pull/5750)
- [**documentation**][**python**] add first batch of methods in oneflow.nn.functional namespace [5693](https://github.com/Oneflow-Inc/oneflow/pull/5693)
- [**purge**][**python**] [part6]Remove singleclient outdated api [5704](https://github.com/Oneflow-Inc/oneflow/pull/5704)
- [**bug**][**python**] use default_generator.seed() as random_seed in init [5721](https://github.com/Oneflow-Inc/oneflow/pull/5721)
- [**bug**][**system**][**python**] ddp broadcast params and buffers [5913](https://github.com/Oneflow-Inc/oneflow/pull/5913)
- [**enhancement**][**test**][**python**] Add consistent tensor requires grad test [5925](https://github.com/Oneflow-Inc/oneflow/pull/5925)
- [**bug**][**python**] wrap flow.nn.init.\* with flow.no_grad() [5932](https://github.com/Oneflow-Inc/oneflow/pull/5932)
- [**feature**][**api**][**python**][**interface**] add clip_grad to optimizer [5817](https://github.com/Oneflow-Inc/oneflow/pull/5817)
- [**enhancement**][**ci**][**op**][**test**][**python**] add randperm with test and docs [5680](https://github.com/Oneflow-Inc/oneflow/pull/5680)
- [**feature**][**api**][**python**] Fea/nn graph/ lr_schedule(and cosine lr_sch) and opt_group [5846](https://github.com/Oneflow-Inc/oneflow/pull/5846)
- [**bug**][**python**] fix bug of SyncOnMasterFn atexit [5909](https://github.com/Oneflow-Inc/oneflow/pull/5909)
- [**purge**][**python**] Delete single client nn modules [6061](https://github.com/Oneflow-Inc/oneflow/pull/6061)
- [**enhancement**][**python**] Move framework.distribute to env [6022](https://github.com/Oneflow-Inc/oneflow/pull/6022)
- [**bug**][**python**] skip sync when abnormally exiting [6025](https://github.com/Oneflow-Inc/oneflow/pull/6025)
- [**feature**][**python**] Fea/nn graph/warmup amp config [5969](https://github.com/Oneflow-Inc/oneflow/pull/5969)
- [**documentation**][**python**] add optimizer api docs [6131](https://github.com/Oneflow-Inc/oneflow/pull/6131)
- [**documentation**][**python**] add_tensor_api_doc [6127](https://github.com/Oneflow-Inc/oneflow/pull/6127)
- [**bug**][**python**] Fix test_grid_sample.py and test_affine_grid.py threshold [6125](https://github.com/Oneflow-Inc/oneflow/pull/6125)
- [**documentation**][**api**][**python**] add doc of graph [6093](https://github.com/Oneflow-Inc/oneflow/pull/6093)
- [**bug**][**python**] Fix make of_format fail in ubuntu [6120](https://github.com/Oneflow-Inc/oneflow/pull/6120)
- [**feature**][**api**][**python**][**interface**] Fea/graph helpers [6088](https://github.com/Oneflow-Inc/oneflow/pull/6088)
- [**enhancement**][**eager**][**python**][**interface**] Use flow.randint in dataloader [6086](https://github.com/Oneflow-Inc/oneflow/pull/6086)
- [**feature**][**eager**][**api**][**python**][**interface**] Import oneflow as torch [6076](https://github.com/Oneflow-Inc/oneflow/pull/6076)
- [**enhancement**][**test**][**api**][**python**][**refactor**] rename OfrecordReader to OFRcordReader [6090](https://github.com/Oneflow-Inc/oneflow/pull/6090)
- [**purge**][**python**][**need-single-client-tests**] Delete single client nn modules [6082](https://github.com/Oneflow-Inc/oneflow/pull/6082)
- [**enhancement**][**python**] flow.load tolerates FileNotFound fault [6083](https://github.com/Oneflow-Inc/oneflow/pull/6083)
- [**feature**][**python**] Fea/pipeline in graph [6105](https://github.com/Oneflow-Inc/oneflow/pull/6105)
- [**enhancement**][**test**][**python**] graph activation checkpointing [6192](https://github.com/Oneflow-Inc/oneflow/pull/6192)
- [**enhancement**][**feature**][**op**][**python**] rnn test [6165](https://github.com/Oneflow-Inc/oneflow/pull/6165)

New in Ops:

- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part2: Add partial unary and math functional apis [5218](https://github.com/Oneflow-Inc/oneflow/pull/5218)
- [**enhancement**][**bug**][**op**][**interface**] Refine deconv kernel [5229](https://github.com/Oneflow-Inc/oneflow/pull/5229)
- [**enhancement**][**op**][**api**][**interface**] add ReflectionPad2d [5172](https://github.com/Oneflow-Inc/oneflow/pull/5172)
- [**feature**][**eager**][**op**][**api**][**interface**] crossentropyloss and nllloss support ignore_index [5195](https://github.com/Oneflow-Inc/oneflow/pull/5195)
- [**feature**][**eager**][**op**][**api**][**interface**] Yejiaojiao/dev bcewithlogitsloss [5173](https://github.com/Oneflow-Inc/oneflow/pull/5173)
- [**bug**][**ci**][**op**] Dev user op set default is_dynamic [5223](https://github.com/Oneflow-Inc/oneflow/pull/5223)
- [**enhancement**][**op**] add magic method for pow [5199](https://github.com/Oneflow-Inc/oneflow/pull/5199)
- [**enhancement**][**op**][**interface**] add cpu version of upsampling [5194](https://github.com/Oneflow-Inc/oneflow/pull/5194)
- [**enhancement**][**bug**][**op**][**api**][**interface**] add ReplicationPad2d [5148](https://github.com/Oneflow-Inc/oneflow/pull/5148)
- [**feature**][**eager**][**op**][**api**][**interface**] add kldivloss module [5155](https://github.com/Oneflow-Inc/oneflow/pull/5155)
- [**feature**][**eager**][**op**][**documentation**][**build**][**api**][**interface**] Add floor module and the corresponding testcases [4964](https://github.com/Oneflow-Inc/oneflow/pull/4964)
- [**enhancement**][**feature**][**op**] Dev conv1d module [5280](https://github.com/Oneflow-Inc/oneflow/pull/5280)
- [**enhancement**][**op**] Add ctc_greedy_decoder op [5294](https://github.com/Oneflow-Inc/oneflow/pull/5294)
- [**enhancement**][**op**][**system**] Dev remove default grad func [5320](https://github.com/Oneflow-Inc/oneflow/pull/5320)
- [**enhancement**][**op**][**system**] Add pad grad func. [5354](https://github.com/Oneflow-Inc/oneflow/pull/5354)
- [**enhancement**][**op**][**system**] Add gradient funcs. [5348](https://github.com/Oneflow-Inc/oneflow/pull/5348)
- [**feature**][**purge**][**bug**][**eager**][**op**][**interface**] fix upsample nearest bug [5347](https://github.com/Oneflow-Inc/oneflow/pull/5347)
- [**enhancement**][**op**][**system**] [Functional] Part7: Migrate pooling ops [5253](https://github.com/Oneflow-Inc/oneflow/pull/5253)
- [**enhancement**][**op**] nvjpeg hardware acc [5240](https://github.com/Oneflow-Inc/oneflow/pull/5240)
- [**enhancement**][**feature**][**ci**][**eager**][**op**][**api**][**interface**] Add bmm module [5334](https://github.com/Oneflow-Inc/oneflow/pull/5334)
- [**enhancement**][**eager**][**op**] Dev image decode eager [5333](https://github.com/Oneflow-Inc/oneflow/pull/5333)
- [**enhancement**][**op**] Optimize softmax warp impl [4977](https://github.com/Oneflow-Inc/oneflow/pull/4977)
- [**enhancement**][**eager**][**op**] Dev tensor buffer eager [5317](https://github.com/Oneflow-Inc/oneflow/pull/5317)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part6: Migrate conv op [5252](https://github.com/Oneflow-Inc/oneflow/pull/5252)
- [**enhancement**][**eager**][**op**] Dev sort eager [5284](https://github.com/Oneflow-Inc/oneflow/pull/5284)
- [**enhancement**][**bug**][**op**][**api**] fix bceloss bug in default weight and reduction [5303](https://github.com/Oneflow-Inc/oneflow/pull/5303)
- [**bug**][**eager**][**op**] remove redundant assert and check [5264](https://github.com/Oneflow-Inc/oneflow/pull/5264)
- [**enhancement**][**bug**][**ci**][**op**] fix bceloss bug about weight [5269](https://github.com/Oneflow-Inc/oneflow/pull/5269)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part5: Migrate nn ops [5249](https://github.com/Oneflow-Inc/oneflow/pull/5249)
- [**enhancement**][**eager**][**op**] Dev argsort eager [5273](https://github.com/Oneflow-Inc/oneflow/pull/5273)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part4: Migrate array ops [5247](https://github.com/Oneflow-Inc/oneflow/pull/5247)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part3: Migrate binary and activation ops [5246](https://github.com/Oneflow-Inc/oneflow/pull/5246)
- [**bug**][**ci**][**op**][**test**] Dev fix rmsprop ci fail [5481](https://github.com/Oneflow-Inc/oneflow/pull/5481)
- [**enhancement**][**op**] add inplace method: Tensor.sin\_ [5471](https://github.com/Oneflow-Inc/oneflow/pull/5471)
- [**bug**][**op**] hotfix image_batch_align [5461](https://github.com/Oneflow-Inc/oneflow/pull/5461)
- [**enhancement**][**eager**][**op**][**interface**] Dev maxpool series op 123d [5244](https://github.com/Oneflow-Inc/oneflow/pull/5244)
- [**bug**][**op**] fix pool gpu kernel [5446](https://github.com/Oneflow-Inc/oneflow/pull/5446)
- [**feature**][**eager**][**op**][**api**][**interface**] add pixelshufflev2 module [5383](https://github.com/Oneflow-Inc/oneflow/pull/5383)
- [**enhancement**][**feature**][**ci**][**eager**][**op**][**documentation**][**api**][**interface**] Add flow xxx and tensor xxx autotest [5386](https://github.com/Oneflow-Inc/oneflow/pull/5386)
- [**enhancement**][**feature**][**eager**][**op**][**api**][**interface**] Modules chunk [5324](https://github.com/Oneflow-Inc/oneflow/pull/5324)
- [**enhancement**][**eager**][**op**] add image normalize for eager [5402](https://github.com/Oneflow-Inc/oneflow/pull/5402)
- [**enhancement**][**eager**][**op**] Dev batch align module [5401](https://github.com/Oneflow-Inc/oneflow/pull/5401)
- [**enhancement**][**eager**][**op**] add coco reader module [5391](https://github.com/Oneflow-Inc/oneflow/pull/5391)
- [**enhancement**][**wip**][**op**] Restruct Elementwise kernel [4130](https://github.com/Oneflow-Inc/oneflow/pull/4130)
- [**bug**][**op**] Fix DecodeRandom reuse mem [5606](https://github.com/Oneflow-Inc/oneflow/pull/5606)
- [**enhancement**][**op**] Align pytorch maxpool [5525](https://github.com/Oneflow-Inc/oneflow/pull/5525)
- [**enhancement**][**bottleneck**][**eager**][**op**][**api**] implementation of constantpad-3d op [5529](https://github.com/Oneflow-Inc/oneflow/pull/5529)
- [**enhancement**][**eager**][**op**] Add scale size for resize [5509](https://github.com/Oneflow-Inc/oneflow/pull/5509)
- [**enhancement**][**op**][**api**][**refactor**] Dev optimize tensor setitem [5501](https://github.com/Oneflow-Inc/oneflow/pull/5501)
- [**enhancement**][**op**] register uint8 dtypeto support dataloader [5499](https://github.com/Oneflow-Inc/oneflow/pull/5499)
- [**enhancement**][**op**] Add unique.cuh [5487](https://github.com/Oneflow-Inc/oneflow/pull/5487)
- [**enhancement**][**op**][**api**][**interface**] Dev ofrecord auto truncating [5412](https://github.com/Oneflow-Inc/oneflow/pull/5412)
- [**feature**][**op**][**system**][**interface**] Feat: LazyInterpret::ApplyImpl support SourceUserOpExpr and Copy [5711](https://github.com/Oneflow-Inc/oneflow/pull/5711)
- [**enhancement**][**op**][**interface**] Dev logical_and/or modules [5636](https://github.com/Oneflow-Inc/oneflow/pull/5636)
- [**enhancement**][**op**] support any number positional arguments for ones and zeros op [5698](https://github.com/Oneflow-Inc/oneflow/pull/5698)
- [**enhancement**][**feature**][**eager**][**op**] Add conv3d Module [5327](https://github.com/Oneflow-Inc/oneflow/pull/5327)
- [**feature**][**eager**][**op**][**api**][**interface**] add batchnorm3d module [5631](https://github.com/Oneflow-Inc/oneflow/pull/5631)
- [**bug**][**eager**][**op**] fix reduce min max backward bug [5651](https://github.com/Oneflow-Inc/oneflow/pull/5651)
- [**enhancement**][**op**] Debug dim scatter [5371](https://github.com/Oneflow-Inc/oneflow/pull/5371)
- [**enhancement**][**op**][**interface**] Dev eye [5583](https://github.com/Oneflow-Inc/oneflow/pull/5583)
- [**enhancement**][**eager**][**op**] Dev minimum maximum [5576](https://github.com/Oneflow-Inc/oneflow/pull/5576)
- [**enhancement**][**op**] Restruct activation grad op [5669](https://github.com/Oneflow-Inc/oneflow/pull/5669)
- [**enhancement**][**feature**][**eager**][**op**] Rewrite activation function [5465](https://github.com/Oneflow-Inc/oneflow/pull/5465)
- [**bug**][**op**][**documentation**] add oneflow.cat for documentation [5621](https://github.com/Oneflow-Inc/oneflow/pull/5621)
- [**enhancement**][**op**] Lcy logsoftmax [5746](https://github.com/Oneflow-Inc/oneflow/pull/5746)
- [**feature**][**op**][**need-simple-ci**] Feat empty op [5659](https://github.com/Oneflow-Inc/oneflow/pull/5659)
- [**enhancement**][**eager**][**op**] Dev split [5714](https://github.com/Oneflow-Inc/oneflow/pull/5714)
- [**enhancement**][**op**][**interface**] add index_select op [5661](https://github.com/Oneflow-Inc/oneflow/pull/5661)
- [**bug**][**op**] fix nvjpeg hw acc [5851](https://github.com/Oneflow-Inc/oneflow/pull/5851)
- [**enhancement**][**op**] Remove move in conv_cudnn [5828](https://github.com/Oneflow-Inc/oneflow/pull/5828)
- [**enhancement**][**op**][**interface**] Dev logical_xor module [5694](https://github.com/Oneflow-Inc/oneflow/pull/5694)
- [**bug**][**eager**][**op**] fix squeeze [5808](https://github.com/Oneflow-Inc/oneflow/pull/5808)
- [**enhancement**][**op**] Get parallel_id and parallel_num through rank and world size in DDP [5717](https://github.com/Oneflow-Inc/oneflow/pull/5717)
- [**bug**][**eager**][**op**] delete interpolate int type [5805](https://github.com/Oneflow-Inc/oneflow/pull/5805)
- [**bug**][**op**] Fix bug in scatter [5743](https://github.com/Oneflow-Inc/oneflow/pull/5743)
- [**enhancement**][**op**] Refactor: remove module not required, call function directly [5754](https://github.com/Oneflow-Inc/oneflow/pull/5754)
- [**enhancement**][**op**] Remove modules not required(tan, erfc, log1p, scatter_nd) [5791](https://github.com/Oneflow-Inc/oneflow/pull/5791)
- [**enhancement**][**op**] Refactor scatter, clamp and pow in cpp instead of in python [5715](https://github.com/Oneflow-Inc/oneflow/pull/5715)
- [**enhancement**][**op**] Rm useless code in gather files [5687](https://github.com/Oneflow-Inc/oneflow/pull/5687)
- [**enhancement**][**eager**][**op**] change flip_code to scalar [5786](https://github.com/Oneflow-Inc/oneflow/pull/5786)
- [**enhancement**][**bug**][**op**][**interface**] fix upsample bug [5753](https://github.com/Oneflow-Inc/oneflow/pull/5753)
- [**bug**][**op**][**interface**] Quick fix Lazy nn.Graph input/output OpConf.BlobConf.is_dynamic [5767](https://github.com/Oneflow-Inc/oneflow/pull/5767)
- [**enhancement**][**bug**][**eager**][**op**] fix argwhere 0-dim bug [5760](https://github.com/Oneflow-Inc/oneflow/pull/5760)
- [**enhancement**][**eager**][**op**] delete unused code [5744](https://github.com/Oneflow-Inc/oneflow/pull/5744)
- [**feature**][**op**] Export fused_scale_tril op [5933](https://github.com/Oneflow-Inc/oneflow/pull/5933)
- [**bug**][**op**] Fix backward bug in 3d [5908](https://github.com/Oneflow-Inc/oneflow/pull/5908)
- [**bug**][**op**] Fix one_hot api limit [5927](https://github.com/Oneflow-Inc/oneflow/pull/5927)
- [**enhancement**][**eager**][**op**] Dev where scalar [5797](https://github.com/Oneflow-Inc/oneflow/pull/5797)
- [**bug**][**op**] fix grad error [5914](https://github.com/Oneflow-Inc/oneflow/pull/5914)
- [**feature**][**bug**][**op**] Fix inplace op circle reference bug [5910](https://github.com/Oneflow-Inc/oneflow/pull/5910)
- [**enhancement**][**op**] Move the judgment content to c++， And add scalar fmod [5854](https://github.com/Oneflow-Inc/oneflow/pull/5854)
- [**enhancement**][**op**] Support combined_margin_loss op in flow.nn.modules [5830](https://github.com/Oneflow-Inc/oneflow/pull/5830)
- [**enhancement**][**op**][**api**][**interface**] functional_one_hot [5315](https://github.com/Oneflow-Inc/oneflow/pull/5315)
- [**enhancement**][**op**] Dev scalar op [5778](https://github.com/Oneflow-Inc/oneflow/pull/5778)
- [**bug**][**eager**][**op**] fix gather kernel 0 shape [5888](https://github.com/Oneflow-Inc/oneflow/pull/5888)
- [**enhancement**][**op**] add l2_normalize for mutl-client interfaces [5859](https://github.com/Oneflow-Inc/oneflow/pull/5859)
- [**feature**][**op**] Export function softmax_cross_entropy [6056](https://github.com/Oneflow-Inc/oneflow/pull/6056)
- [**enhancement**][**op**] Add int attr for functional adaptive average pool [6059](https://github.com/Oneflow-Inc/oneflow/pull/6059)
- [**enhancement**][**op**][**interface**] dev full op [5955](https://github.com/Oneflow-Inc/oneflow/pull/5955)
- [**bug**][**eager**][**op**] fix 0dim inplace add [6029](https://github.com/Oneflow-Inc/oneflow/pull/6029)
- [**feature**][**op**][**system**][**interface**] Feat: nn.Graph image gpu decoder [6014](https://github.com/Oneflow-Inc/oneflow/pull/6014)
- [**enhancement**][**op**][**interface**] dev optim_optim_lr_scheduler_multisteplr [5975](https://github.com/Oneflow-Inc/oneflow/pull/5975)
- [**enhancement**][**op**] NopKernel [6035](https://github.com/Oneflow-Inc/oneflow/pull/6035)
- [**enhancement**][**eager**][**op**][**api**] Dev tril op [6005](https://github.com/Oneflow-Inc/oneflow/pull/6005)
- [**enhancement**][**op**] dev unfold and fold [5675](https://github.com/Oneflow-Inc/oneflow/pull/5675)
- [**enhancement**][**op**] ResNet CUDA Graphs [6018](https://github.com/Oneflow-Inc/oneflow/pull/6018)
- [**enhancement**][**feature**][**op**] add broadcast pow [6013](https://github.com/Oneflow-Inc/oneflow/pull/6013)
- [**enhancement**][**op**][**interface**] init of op diag [5298](https://github.com/Oneflow-Inc/oneflow/pull/5298)
- [**op**][**documentation**][**api**] Fix api document bug [6009](https://github.com/Oneflow-Inc/oneflow/pull/6009)
- [**enhancement**][**op**] Dev fused functional [5954](https://github.com/Oneflow-Inc/oneflow/pull/5954)
- [**bug**][**op**][**build**] Add nvcc flag -Werror cross-execution-space-call [6002](https://github.com/Oneflow-Inc/oneflow/pull/6002)
- [**bug**][**op**] Fix Normalization grad function [5993](https://github.com/Oneflow-Inc/oneflow/pull/5993)
- [**enhancement**][**feature**][**eager**][**op**][**test**][**interface**] Add fused self attention [5966](https://github.com/Oneflow-Inc/oneflow/pull/5966)
- [**enhancement**][**bug**][**ci**][**eager**][**op**][**api**][**interface**] Try to fix var bug [5973](https://github.com/Oneflow-Inc/oneflow/pull/5973)
- [**enhancement**][**feature**][**eager**][**op**][**interface**] add prod op [5867](https://github.com/Oneflow-Inc/oneflow/pull/5867)
- [**enhancement**][**eager**][**op**][**api**] add glu op [6065](https://github.com/Oneflow-Inc/oneflow/pull/6065)
- [**enhancement**][**op**] Align Torch.nn.functional poolXd [6184](https://github.com/Oneflow-Inc/oneflow/pull/6184)
- [**bug**][**eager**][**op**] fix backward index for gamma beta [6149](https://github.com/Oneflow-Inc/oneflow/pull/6149)
- [**bug**][**op**][**system**] Fix BroadcastMatmulGrad bug [6168](https://github.com/Oneflow-Inc/oneflow/pull/6168)
- [**enhancement**][**op**][**api**] Add Int support for functional.avg/maxpool [6174](https://github.com/Oneflow-Inc/oneflow/pull/6174)
- [**bug**][**eager**][**op**][**api**][**interface**] align dropout api name with pytorch [6170](https://github.com/Oneflow-Inc/oneflow/pull/6170)
- [**enhancement**][**op**] support inplace operation for hardsigmoid [6137](https://github.com/Oneflow-Inc/oneflow/pull/6137)
- [**enhancement**][**bug**][**op**] Fix do bias correction in Adam/AdamW [5960](https://github.com/Oneflow-Inc/oneflow/pull/5960)
- [**bug**][**eager**][**op**][**api**][**interface**] fix repeat 0-dim tensor bug [6150](https://github.com/Oneflow-Inc/oneflow/pull/6150)
- [**enhancement**][**bug**][**op**] Fix select_first_grad bug [6142](https://github.com/Oneflow-Inc/oneflow/pull/6142)
- [**bug**][**ci**][**eager**][**op**][**documentation**][**interface**] Add clipgrad doc and contiguous [6130](https://github.com/Oneflow-Inc/oneflow/pull/6130)
- [**bug**][**op**] Fix eager optim dynamic attr bug [6111](https://github.com/Oneflow-Inc/oneflow/pull/6111)
- [**enhancement**][**op**] Support grid_sample and affine_grid operator [6038](https://github.com/Oneflow-Inc/oneflow/pull/6038)
- [**op**][**documentation**] Export apis for documentation [6068](https://github.com/Oneflow-Inc/oneflow/pull/6068)
- [**enhancement**][**feature**][**bug**][**ci**][**eager**][**op**][**documentation**][**interface**] transfer python function to c++ method [6114](https://github.com/Oneflow-Inc/oneflow/pull/6114)
- [**op**][**documentation**] Dev functional batch_gather [6233](https://github.com/Oneflow-Inc/oneflow/pull/6233)
- [**enhancement**][**op**][**test**] fix cross_entropy_loss and its test [5799](https://github.com/Oneflow-Inc/oneflow/pull/5799)
- [**bug**][**op**] Use attr nd_sbp to check consistent [6222](https://github.com/Oneflow-Inc/oneflow/pull/6222)
- [**enhancement**][**op**] Dev fused bn functional [6077](https://github.com/Oneflow-Inc/oneflow/pull/6077)
- [**enhancement**][**op**] support default value in intlist [6201](https://github.com/Oneflow-Inc/oneflow/pull/6201)
- [**bug**][**op**] fix sparse_softmax get_nd_sbp [6203](https://github.com/Oneflow-Inc/oneflow/pull/6203)
- [**bug**][**op**] Fix bug in model fused update [6197](https://github.com/Oneflow-Inc/oneflow/pull/6197)
- [**enhancement**][**op**][**system**][**refactor**] Optimize tensor getitem. [5433](https://github.com/Oneflow-Inc/oneflow/pull/5433)

New in Eager:

- [**enhancement**][**eager**][**interface**] Reconstruct module files [5251](https://github.com/Oneflow-Inc/oneflow/pull/5251)
- [**bug**][**eager**][**documentation**][**interface**] Fix conv module bug [5245](https://github.com/Oneflow-Inc/oneflow/pull/5245)
- [**bug**][**ci**][**eager**][**interface**] Fix bce withlogitloss ci error [5237](https://github.com/Oneflow-Inc/oneflow/pull/5237)
- [**feature**][**eager**][**api**][**interface**] module BCELoss [5144](https://github.com/Oneflow-Inc/oneflow/pull/5144)
- [**enhancement**][**feature**][**eager**][**api**][**interface**] Dev norm op [5178](https://github.com/Oneflow-Inc/oneflow/pull/5178)
- [**enhancement**][**bug**][**eager**] Fix stack module [5222](https://github.com/Oneflow-Inc/oneflow/pull/5222)
- [**enhancement**][**feature**][**eager**] Support different dtype of equal module [5214](https://github.com/Oneflow-Inc/oneflow/pull/5214)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**][**interface**] Add nllloss backward [5210](https://github.com/Oneflow-Inc/oneflow/pull/5210)
- [**enhancement**][**eager**][**api**][**upload-core**] Decouple FileSystem and IOConf [5162](https://github.com/Oneflow-Inc/oneflow/pull/5162)
- [**enhancement**][**ci**][**eager**] Set lower precision avoid ci failing [5200](https://github.com/Oneflow-Inc/oneflow/pull/5200)
- [**eager**][**documentation**] Add hint when apply FunctionNode second time [5369](https://github.com/Oneflow-Inc/oneflow/pull/5369)
- [**enhancement**][**feature**][**bug**][**ci**][**eager**][**documentation**][**api**] Fix upsample bilinear bug [5366](https://github.com/Oneflow-Inc/oneflow/pull/5366)
- [**bug**][**eager**] Fix not contiguous ndarray to tensor bug [5351](https://github.com/Oneflow-Inc/oneflow/pull/5351)
- [**enhancement**][**eager**][**system**] Infer consistent tensor meta [5118](https://github.com/Oneflow-Inc/oneflow/pull/5118)
- [**feature**][**eager**] Feat graph autograd engine [5296](https://github.com/Oneflow-Inc/oneflow/pull/5296)
- [**enhancement**][**eager**][**interface**] Dev type as module [5349](https://github.com/Oneflow-Inc/oneflow/pull/5349)
- [**feature**][**eager**][**documentation**][**api**][**interface**] Add new ones module [5342](https://github.com/Oneflow-Inc/oneflow/pull/5342)
- [**enhancement**][**bug**][**eager**] Fix logical slice assign dtype [5339](https://github.com/Oneflow-Inc/oneflow/pull/5339)
- [**bug**][**ci**][**eager**][**documentation**][**api**][**interface**] Fix where module bug [5300](https://github.com/Oneflow-Inc/oneflow/pull/5300)
- [**bug**][**ci**][**eager**][**documentation**][**api**] Fix l1loss ci error [5307](https://github.com/Oneflow-Inc/oneflow/pull/5307)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**][**interface**] Qi's First Edit of deleting "print" and ".numpy" [5129](https://github.com/Oneflow-Inc/oneflow/pull/5129)
- [**feature**][**eager**][**refactor**] Separate autograd meta to tensor [5267](https://github.com/Oneflow-Inc/oneflow/pull/5267)
- [**feature**][**eager**][**api**][**interface**] add tile module [5234](https://github.com/Oneflow-Inc/oneflow/pull/5234)
- [**enhancement**][**eager**] Release lambda function to reuse tensor memory [5266](https://github.com/Oneflow-Inc/oneflow/pull/5266)
- [**feature**][**bug**][**eager**][**documentation**] Fix default value not set bug [5483](https://github.com/Oneflow-Inc/oneflow/pull/5483)
- [**enhancement**][**eager**][**interface**] [Add] gather_nd scatter_nd [5422](https://github.com/Oneflow-Inc/oneflow/pull/5422)
- [**enhancement**][**bug**][**eager**] fix param [5473](https://github.com/Oneflow-Inc/oneflow/pull/5473)
- [**bug**][**eager**] Fix Tensor.grad setter bug [5462](https://github.com/Oneflow-Inc/oneflow/pull/5462)
- [**enhancement**][**eager**] Rename now_grad_arg to current_grad [5466](https://github.com/Oneflow-Inc/oneflow/pull/5466)
- [**eager**][**test**][**documentation**][**interface**] Add autotest part1 [5436](https://github.com/Oneflow-Inc/oneflow/pull/5436)
- [**enhancement**][**eager**] Use functional copy instead of op_builder [5460](https://github.com/Oneflow-Inc/oneflow/pull/5460)
- [**bottleneck**][**bug**][**eager**][**interface**] fix -1 index not support bug [5448](https://github.com/Oneflow-Inc/oneflow/pull/5448)
- [**bug**][**ci**][**eager**][**documentation**][**api**] Fix concat backward bug [5443](https://github.com/Oneflow-Inc/oneflow/pull/5443)
- [**enhancement**][**bug**][**ci**][**eager**] Add autograd engine warning [5444](https://github.com/Oneflow-Inc/oneflow/pull/5444)
- [**feature**][**eager**][**api**][**interface**] Smoothl1loss [5256](https://github.com/Oneflow-Inc/oneflow/pull/5256)
- [**enhancement**][**bottleneck**][**eager**] remove device dtype params [5434](https://github.com/Oneflow-Inc/oneflow/pull/5434)
- [**bug**][**ci**][**eager**][**documentation**][**interface**] Delete maxpool failed test [5409](https://github.com/Oneflow-Inc/oneflow/pull/5409)
- [**enhancement**][**eager**][**api**] Add tensor grad assginment [5379](https://github.com/Oneflow-Inc/oneflow/pull/5379)
- [**enhancement**][**bug**][**eager**] fix-abs [5398](https://github.com/Oneflow-Inc/oneflow/pull/5398)
- [**enhancement**][**bug**][**eager**][**interface**] Fix bn track running stats [5393](https://github.com/Oneflow-Inc/oneflow/pull/5393)
- [**enhancement**][**bug**][**eager**] Support uint dtype of constant op [5396](https://github.com/Oneflow-Inc/oneflow/pull/5396)
- [**enhancement**][**bug**][**eager**][**documentation**][**interface**] Delete useless code upsample [5392](https://github.com/Oneflow-Inc/oneflow/pull/5392)
- [**enhancement**][**ci**][**eager**][**interface**] add flow.view [5301](https://github.com/Oneflow-Inc/oneflow/pull/5301)
- [**enhancement**][**bug**][**ci**][**eager**][**api**][**interface**] Add masked select module [5356](https://github.com/Oneflow-Inc/oneflow/pull/5356)
- [**bug**][**eager**][**interface**] Fix batchnorm backward bug [5602](https://github.com/Oneflow-Inc/oneflow/pull/5602)
- [**enhancement**][**eager**] Support weight_dacay(l2 actually) [5587](https://github.com/Oneflow-Inc/oneflow/pull/5587)
- [**feature**][**eager**][**documentation**][**api**] Add new autotest [5588](https://github.com/Oneflow-Inc/oneflow/pull/5588)
- [**enhancement**][**eager**][**documentation**][**api**] Dev fmod [5404](https://github.com/Oneflow-Inc/oneflow/pull/5404)
- [**feature**][**eager**] Support inplace add [5432](https://github.com/Oneflow-Inc/oneflow/pull/5432)
- [**feature**][**eager**][**interface**] Feat tensor stride property [5543](https://github.com/Oneflow-Inc/oneflow/pull/5543)
- [**enhancement**][**feature**][**eager**][**documentation**][**api**] Add flip module [5541](https://github.com/Oneflow-Inc/oneflow/pull/5541)
- [**feature**][**eager**] Feat module repr [5486](https://github.com/Oneflow-Inc/oneflow/pull/5486)
- [**enhancement**][**bottleneck**][**bug**][**eager**][**interface**] Fix maxpool1d params [5493](https://github.com/Oneflow-Inc/oneflow/pull/5493)
- [**enhancement**][**feature**][**eager**][**interface**] Dev flow.utils.data part1 [5406](https://github.com/Oneflow-Inc/oneflow/pull/5406)
- [**bug**][**eager**][**api**] Fix tensor getitem bug [5474](https://github.com/Oneflow-Inc/oneflow/pull/5474)
- [**enhancement**][**eager**][**need-simple-ci**] export datasets interface [5691](https://github.com/Oneflow-Inc/oneflow/pull/5691)
- [**enhancement**][**eager**][**system**] rebase [5601](https://github.com/Oneflow-Inc/oneflow/pull/5601)
- [**enhancement**][**eager**][**test**] added nn.RecordBytesDecoder with its test [5475](https://github.com/Oneflow-Inc/oneflow/pull/5475)
- [**enhancement**][**feature**][**eager**][**need-simple-ci**] 0-dim tensor support [5552](https://github.com/Oneflow-Inc/oneflow/pull/5552)
- [**enhancement**][**bug**][**eager**] rewrite slice_update backward [5677](https://github.com/Oneflow-Inc/oneflow/pull/5677)
- [**enhancement**][**bug**][**eager**][**interface**] align view input style with torch [5676](https://github.com/Oneflow-Inc/oneflow/pull/5676)
- [**enhancement**][**eager**][**interface**][**need-simple-ci**] add autotests for modules [5666](https://github.com/Oneflow-Inc/oneflow/pull/5666)
- [**enhancement**][**bottleneck**][**eager**][**interface**] Dev constantpad1d op [5579](https://github.com/Oneflow-Inc/oneflow/pull/5579)
- [**enhancement**][**eager**][**api**][**interface**] Restruct MathOps AutoTest [5654](https://github.com/Oneflow-Inc/oneflow/pull/5654)
- [**enhancement**][**bug**][**ci**][**eager**] Fix flip bug [5657](https://github.com/Oneflow-Inc/oneflow/pull/5657)
- [**bug**][**eager**][**api**][**interface**] Fix expand module bug [5650](https://github.com/Oneflow-Inc/oneflow/pull/5650)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**] Fix repeat bug [5633](https://github.com/Oneflow-Inc/oneflow/pull/5633)
- [**enhancement**][**eager**][**test**][**api**][**interface**] Add new autotest [5617](https://github.com/Oneflow-Inc/oneflow/pull/5617)
- [**enhancement**][**eager**][**api**][**interface**] Dev flow.utils.data part2 [5500](https://github.com/Oneflow-Inc/oneflow/pull/5500)
- [**enhancement**][**bug**][**eager**] make setitem device match [5835](https://github.com/Oneflow-Inc/oneflow/pull/5835)
- [**bug**][**eager**][**api**][**interface**] align reshape input param with pytorch [5804](https://github.com/Oneflow-Inc/oneflow/pull/5804)
- [**feature**][**bug**][**eager**][**api**] Align where op with torch [5850](https://github.com/Oneflow-Inc/oneflow/pull/5850)
- [**enhancement**][**bug**][**eager**][**api**] Restruct prelu op [5829](https://github.com/Oneflow-Inc/oneflow/pull/5829)
- [**bug**][**eager**][**need-simple-ci**] fix pooling ceil_mode bug [5818](https://github.com/Oneflow-Inc/oneflow/pull/5818)
- [**enhancement**][**eager**] stateful local kernel supports consistent [5789](https://github.com/Oneflow-Inc/oneflow/pull/5789)
- [**bug**][**eager**][**api**][**interface**] Fix argwhere bug [5816](https://github.com/Oneflow-Inc/oneflow/pull/5816)
- [**enhancement**][**eager**][**documentation**][**api**] dev-nonzero [5809](https://github.com/Oneflow-Inc/oneflow/pull/5809)
- [**enhancement**][**feature**][**eager**][**api**] Add fake quantize op [5690](https://github.com/Oneflow-Inc/oneflow/pull/5690)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**] Add api [5663](https://github.com/Oneflow-Inc/oneflow/pull/5663)
- [**enhancement**][**eager**] Refactor consistent infer result [5790](https://github.com/Oneflow-Inc/oneflow/pull/5790)
- [**bug**][**eager**][**need-simple-ci**] skip dataloader test [5780](https://github.com/Oneflow-Inc/oneflow/pull/5780)
- [**bug**][**eager**][**need-simple-ci**] fix 0-dim tensor.fill\_ [5771](https://github.com/Oneflow-Inc/oneflow/pull/5771)
- [**enhancement**][**eager**] Cpu mpi broadcast [5726](https://github.com/Oneflow-Inc/oneflow/pull/5726)
- [**feature**][**eager**] Feat grad mode classes [5956](https://github.com/Oneflow-Inc/oneflow/pull/5956)
- [**enhancement**][**bug**][**eager**] fix wrong names [5951](https://github.com/Oneflow-Inc/oneflow/pull/5951)
- [**enhancement**][**eager**][**system**] Local dep object pool [5953](https://github.com/Oneflow-Inc/oneflow/pull/5953)
- [**enhancement**][**eager**][**interface**] rename OpExprInterpState to AutoGradCaptureState [5918](https://github.com/Oneflow-Inc/oneflow/pull/5918)
- [**bug**][**eager**] Fix linear bug [5945](https://github.com/Oneflow-Inc/oneflow/pull/5945)
- [**bug**][**eager**] Fix tensor_meta update bug [5924](https://github.com/Oneflow-Inc/oneflow/pull/5924)
- [**enhancement**][**eager**] use flow.randperm [5928](https://github.com/Oneflow-Inc/oneflow/pull/5928)
- [**enhancement**][**eager**] consistent init/save/load [5896](https://github.com/Oneflow-Inc/oneflow/pull/5896)
- [**enhancement**][**bug**][**eager**][**documentation**][**interface**] Restruct sort and argsort op [5911](https://github.com/Oneflow-Inc/oneflow/pull/5911)
- [**enhancement**][**bug**][**eager**][**interface**] Try to fix the problem that the insightface cannot converge。 [5906](https://github.com/Oneflow-Inc/oneflow/pull/5906)
- [**enhancement**][**bug**][**eager**][**interface**] Add autotest [5899](https://github.com/Oneflow-Inc/oneflow/pull/5899)
- [**enhancement**][**eager**] The scheduler thread joins worker threads [5893](https://github.com/Oneflow-Inc/oneflow/pull/5893)
- [**enhancement**][**eager**] Bugfix async callback [5881](https://github.com/Oneflow-Inc/oneflow/pull/5881)
- [**feature**][**eager**] Feat tensor to bool [5836](https://github.com/Oneflow-Inc/oneflow/pull/5836)
- [**bug**][**eager**] Remove inplace broadcast_add [5551](https://github.com/Oneflow-Inc/oneflow/pull/5551)
- [**enhancement**][**eager**] Broadcast consistent shape and dtype [5784](https://github.com/Oneflow-Inc/oneflow/pull/5784)
- [**enhancement**][**eager**] Fix optimizer list parameters input bug [5848](https://github.com/Oneflow-Inc/oneflow/pull/5848)
- [**enhancement**][**eager**][**interface**] Dev flow.utils.data part3 [5644](https://github.com/Oneflow-Inc/oneflow/pull/5644)
- [**enhancement**][**eager**][**api**] Normalize naming of modules [6066](https://github.com/Oneflow-Inc/oneflow/pull/6066)
- [**enhancement**][**feature**][**eager**][**api**][**interface**] add trunc*normal* [6051](https://github.com/Oneflow-Inc/oneflow/pull/6051)
- [**enhancement**][**bug**][**eager**] AutoMatedTest support test module.parameter.grad [6043](https://github.com/Oneflow-Inc/oneflow/pull/6043)
- [**enhancement**][**feature**][**bug**][**eager**] add module **call** kwags [6069](https://github.com/Oneflow-Inc/oneflow/pull/6069)
- [**enhancement**][**eager**][**api**][**interface**] add tensor.item tensor.tolist [6021](https://github.com/Oneflow-Inc/oneflow/pull/6021)
- [**enhancement**][**eager**][**api**][**interface**] Export pool ops api [6047](https://github.com/Oneflow-Inc/oneflow/pull/6047)
- [**enhancement**][**bug**][**eager**][**test**][**documentation**][**interface**] Add more autotest sample [6039](https://github.com/Oneflow-Inc/oneflow/pull/6039)
- [**enhancement**][**bug**][**eager**][**system**] disable cuda_h2d stream [6020](https://github.com/Oneflow-Inc/oneflow/pull/6020)
- [**feature**][**eager**][**test**][**api**][**interface**] Add autotest codegen [6019](https://github.com/Oneflow-Inc/oneflow/pull/6019)
- [**feature**][**eager**][**documentation**] Refactor cosine lr scheduler [6000](https://github.com/Oneflow-Inc/oneflow/pull/6000)
- [**enhancement**][**eager**][**interface**] tensor.cpu/tensor.cuda [5894](https://github.com/Oneflow-Inc/oneflow/pull/5894)
- [**enhancement**][**eager**][**api**] Support consistent_tensor.to(dtype) [5991](https://github.com/Oneflow-Inc/oneflow/pull/5991)
- [**bug**][**eager**][**interface**] remove redundant codes in ModuleDict [5961](https://github.com/Oneflow-Inc/oneflow/pull/5961)
- [**bug**][**eager**] Fix LayerNorm check bug [6196](https://github.com/Oneflow-Inc/oneflow/pull/6196)
- [**enhancement**][**eager**][**api**] Change dropout api [6182](https://github.com/Oneflow-Inc/oneflow/pull/6182)
- [**enhancement**][**good for pr**][**eager**][**api**][**interface**] add: test convert dependency [6023](https://github.com/Oneflow-Inc/oneflow/pull/6023)
- [**enhancement**][**bug**][**eager**][**interface**] Fix autotest codegen bug [6171](https://github.com/Oneflow-Inc/oneflow/pull/6171)
- [**bug**][**eager**] restore instr_local_dep_object_pool_size for nccl [6160](https://github.com/Oneflow-Inc/oneflow/pull/6160)
- [**enhancement**][**eager**][**api**][**interface**] Aligin pooling op functional api names with torch [6163](https://github.com/Oneflow-Inc/oneflow/pull/6163)
- [**feature**][**bug**][**eager**][**api**][**interface**] delete file [6162](https://github.com/Oneflow-Inc/oneflow/pull/6162)
- [**bug**][**eager**] Fix optim load_state_dict bug [6152](https://github.com/Oneflow-Inc/oneflow/pull/6152)
- [**enhancement**][**eager**][**api**] add is_training to dropout functor [6148](https://github.com/Oneflow-Inc/oneflow/pull/6148)
- [**enhancement**][**eager**] Decompose nd sbp boxing [5800](https://github.com/Oneflow-Inc/oneflow/pull/5800)
- [**enhancement**][**eager**] support consistent_tensor.to(copy=True) [6122](https://github.com/Oneflow-Inc/oneflow/pull/6122)
- [**feature**][**eager**] Static grad scaler [6135](https://github.com/Oneflow-Inc/oneflow/pull/6135)
- [**bug**][**eager**] Fix LayerNorm expr bug [6121](https://github.com/Oneflow-Inc/oneflow/pull/6121)
- [**bug**][**eager**][**api**] move numpy c api init in numpy.cpp, make np array contiguous before copying [6117](https://github.com/Oneflow-Inc/oneflow/pull/6117)
- [**enhancement**][**eager**][**refactor**] Remove params from ParamGroup getitem [6096](https://github.com/Oneflow-Inc/oneflow/pull/6096)
- [**enhancement**][**feature**][**eager**] Support tensor and optimizer serialization [6087](https://github.com/Oneflow-Inc/oneflow/pull/6087)
- [**enhancement**][**bug**][**eager**] fix bug about tensor str in nonsymmetric cast and getitem in consist… [6239](https://github.com/Oneflow-Inc/oneflow/pull/6239)
- [**enhancement**][**eager**] Cpu all reduce [5849](https://github.com/Oneflow-Inc/oneflow/pull/5849)
- [**feature**][**eager**] Support assign copy interface [6228](https://github.com/Oneflow-Inc/oneflow/pull/6228)
- [**enhancement**][**eager**][**api**][**interface**] Dev reconstruct pad ops [6223](https://github.com/Oneflow-Inc/oneflow/pull/6223)
- [**enhancement**][**eager**][**api**][**interface**] support flow.cuda.is_available [6124](https://github.com/Oneflow-Inc/oneflow/pull/6124)
- [**bug**][**eager**] make flow.\_C.local_all_reduce sync lanuched [6175](https://github.com/Oneflow-Inc/oneflow/pull/6175)
- [**enhancement**][**eager**] Rename flow to oneflow in user hint [6190](https://github.com/Oneflow-Inc/oneflow/pull/6190)
- [**bug**][**eager**][**tooling**][**test**][**api**][**interface**] Autotest generate input tensor [6206](https://github.com/Oneflow-Inc/oneflow/pull/6206)
- [**enhancement**][**eager**] consistent tensor zeros\_() [6202](https://github.com/Oneflow-Inc/oneflow/pull/6202)
- [**enhancement**][**eager**] Cpu mpi [5865](https://github.com/Oneflow-Inc/oneflow/pull/5865)

Build enhancements:

- [**bug**][**build**] Fix GRPC compilation failure on CMake 3.20 [5255](https://github.com/Oneflow-Inc/oneflow/pull/5255)
- [**bug**][**build**] Refine header file copy [5254](https://github.com/Oneflow-Inc/oneflow/pull/5254)
- [**bug**][**build**] Fix older version CMake doesn't support multiple targets in CLI [5248](https://github.com/Oneflow-Inc/oneflow/pull/5248)
- [**bug**][**build**] Turn off NCCL_STATIC/CUDNN_STATIC when CUDA_STATIC is OFF [5243](https://github.com/Oneflow-Inc/oneflow/pull/5243)
- [**feature**][**build**] Fix support for Ninja and add Ninja build in Simple CI [5236](https://github.com/Oneflow-Inc/oneflow/pull/5236)
- [**enhancement**][**build**] Add cmake option CUDA_STATIC [5164](https://github.com/Oneflow-Inc/oneflow/pull/5164)
- [**bug**][**build**] Fix protobuf debug postfix [5233](https://github.com/Oneflow-Inc/oneflow/pull/5233)
- [**enhancement**][**ci**][**build**] Move default third party dir into build dir [5230](https://github.com/Oneflow-Inc/oneflow/pull/5230)
- [**enhancement**][**build**] Refine protobuf cmake [5216](https://github.com/Oneflow-Inc/oneflow/pull/5216)
- [**enhancement**][**ci**][**build**] Remove transport test main [5215](https://github.com/Oneflow-Inc/oneflow/pull/5215)
- [**enhancement**][**ci**][**build**] Speedup opencv build [5213](https://github.com/Oneflow-Inc/oneflow/pull/5213)
- [**enhancement**][**build**] Support clang [5015](https://github.com/Oneflow-Inc/oneflow/pull/5015)
- [**enhancement**][**documentation**][**build**] Add prefix when creating git archive [5201](https://github.com/Oneflow-Inc/oneflow/pull/5201)
- [**enhancement**][**build**] Add cmake option NCCL_STATIC [5160](https://github.com/Oneflow-Inc/oneflow/pull/5160)
- [**enhancement**][**build**] Refine CMake CUDA version handling [5192](https://github.com/Oneflow-Inc/oneflow/pull/5192)
- [**enhancement**][**build**] Use clang plugin to check Maybe variables are used [5358](https://github.com/Oneflow-Inc/oneflow/pull/5358)
- [**enhancement**][**build**] Add BUILD_BYPRODUCTS for ExternalProject_Add [5316](https://github.com/Oneflow-Inc/oneflow/pull/5316)
- [**enhancement**][**build**] Add cmake init cache to simplify user onboarding [5311](https://github.com/Oneflow-Inc/oneflow/pull/5311)
- [**feature**][**bug**][**build**] Fix macOS support and run macOS build in Simple CI [4947](https://github.com/Oneflow-Inc/oneflow/pull/4947)
- [**enhancement**][**build**] flatbuffers use mirror [5295](https://github.com/Oneflow-Inc/oneflow/pull/5295)
- [**enhancement**][**build**] Don't build test by default [5302](https://github.com/Oneflow-Inc/oneflow/pull/5302)
- [**enhancement**][**build**] Prevent building from scratch when toggle flag BUILD_GIT_VERSION [5259](https://github.com/Oneflow-Inc/oneflow/pull/5259)
- [**enhancement**][**build**] Refine gRPC, glog, gflags cmake for conda [5276](https://github.com/Oneflow-Inc/oneflow/pull/5276)
- [**feature**][**build**] Support XLA with CPU-only [5260](https://github.com/Oneflow-Inc/oneflow/pull/5260)
- [**enhancement**][**ci**][**onnx**][**build**] Remove ONNX from CI [5257](https://github.com/Oneflow-Inc/oneflow/pull/5257)
- [**enhancement**][**build**] Refactor build_wheel to support oneflowinc images [5427](https://github.com/Oneflow-Inc/oneflow/pull/5427)
- [**enhancement**][**build**] Add arg skip_audit in build wheel [5423](https://github.com/Oneflow-Inc/oneflow/pull/5423)
- [**bug**][**build**] hwloc disable shared [5388](https://github.com/Oneflow-Inc/oneflow/pull/5388)
- [**documentation**][**build**] Update readme for autoconf and libtool [5376](https://github.com/Oneflow-Inc/oneflow/pull/5376)
- [**enhancement**][**build**] remove dir python and compatible_single_client_python [5609](https://github.com/Oneflow-Inc/oneflow/pull/5609)
- [**bug**][**build**][**system**] Fix pyyaml version [5594](https://github.com/Oneflow-Inc/oneflow/pull/5594)
- [**enhancement**][**ci**][**build**] force release flags [5574](https://github.com/Oneflow-Inc/oneflow/pull/5574)
- [**bug**][**build**] prevent endless loop [5534](https://github.com/Oneflow-Inc/oneflow/pull/5534)
- [**enhancement**][**build**] Support sccache [5528](https://github.com/Oneflow-Inc/oneflow/pull/5528)
- [**enhancement**][**build**] Add definition for CMAKE_BUILD_TYPE and print cmake_build_type in oneflow doctor [5505](https://github.com/Oneflow-Inc/oneflow/pull/5505)
- [**enhancement**][**ci**][**build**][**need-simple-ci**] Fix macOS for recent changes [5705](https://github.com/Oneflow-Inc/oneflow/pull/5705)
- [**bug**][**build**] fix return type error on gcc 4.8.5 [5660](https://github.com/Oneflow-Inc/oneflow/pull/5660)
- [**enhancement**][**build**] Check CMAKE_BUILD_TYPE [5656](https://github.com/Oneflow-Inc/oneflow/pull/5656)
- [**enhancement**][**build**] add -Werror=return-type [5655](https://github.com/Oneflow-Inc/oneflow/pull/5655)
- [**enhancement**][**build**] Clean and fix for new py dir [5618](https://github.com/Oneflow-Inc/oneflow/pull/5618)
- [**enhancement**][**build**] cmake: disable array-bounds check & treat warnings as errors for pyextobj and oneflow_internal & fix warnings [5838](https://github.com/Oneflow-Inc/oneflow/pull/5838)
- [**bug**][**build**] set CMAKE_BUILD_TYPE to Release if undefined [5842](https://github.com/Oneflow-Inc/oneflow/pull/5842)
- [**enhancement**][**build**][**need-simple-ci**] Fix all warnings & Add option TREAT_WARING_AS_ERROR to cmake [5751](https://github.com/Oneflow-Inc/oneflow/pull/5751)
- [**enhancement**][**build**] add CMAKE_INTERPROCEDURAL_OPTIMIZATION in fast cmake cache [5970](https://github.com/Oneflow-Inc/oneflow/pull/5970)
- [**enhancement**][**build**] add clang tidy target [5957](https://github.com/Oneflow-Inc/oneflow/pull/5957)
- [**bug**][**build**] cmake: fix cmake cache args in opencv [5959](https://github.com/Oneflow-Inc/oneflow/pull/5959)
- [**enhancement**][**build**] Add cmake option USE_SYSTEM_NCCL [5897](https://github.com/Oneflow-Inc/oneflow/pull/5897)
- [**enhancement**][**build**] cmake: include third party headers as system headers to avoid warnings [5879](https://github.com/Oneflow-Inc/oneflow/pull/5879)
- [**enhancement**][**build**] Ignore opencv-python on machine aarch64 [5884](https://github.com/Oneflow-Inc/oneflow/pull/5884)
- [**enhancement**][**build**] enable CMake first class cuda support [5858](https://github.com/Oneflow-Inc/oneflow/pull/5858)
- [**bug**][**build**] Fix compile warning (strict-aliasing) [5872](https://github.com/Oneflow-Inc/oneflow/pull/5872)
- [**enhancement**][**bug**][**build**][**need-simple-ci**] Upgrade gtest and fix some errors raised by clang [6079](https://github.com/Oneflow-Inc/oneflow/pull/6079)
- [**bug**][**ci**][**build**] cmake: fix ninja build in CI [6072](https://github.com/Oneflow-Inc/oneflow/pull/6072)
- [**bug**][**build**] fix files not actually removed when building for multiple python versions [6060](https://github.com/Oneflow-Inc/oneflow/pull/6060)
- [**bug**][**build**][**api**] functional_api: fix build error in mac os [6010](https://github.com/Oneflow-Inc/oneflow/pull/6010)
- [**bug**][**build**][**need-simple-ci**][**need-single-client-tests**] Fix recompile from scratch [6036](https://github.com/Oneflow-Inc/oneflow/pull/6036)
- [**bug**][**build**] Turn on NVCC's warnings [6011](https://github.com/Oneflow-Inc/oneflow/pull/6011)
- [**bug**][**build**][**need-single-client-tests**] fix bundle .so of other python version [6034](https://github.com/Oneflow-Inc/oneflow/pull/6034)
- [**bug**][**ci**][**build**][**need-single-client-tests**] use copy_all_files_in_dir to replace copy_files [6033](https://github.com/Oneflow-Inc/oneflow/pull/6033)
- [**enhancement**][**build**] check compiler version in cmake [6026](https://github.com/Oneflow-Inc/oneflow/pull/6026)
- [**enhancement**][**build**] Add CUDA_NVCC_THREADS_NUMBER [6017](https://github.com/Oneflow-Inc/oneflow/pull/6017)
- [**enhancement**][**build**][**need-simple-ci**] optimize of_include_copy [5978](https://github.com/Oneflow-Inc/oneflow/pull/5978)
- [**enhancement**][**ci**][**build**][**need-single-client-tests**] CI: remove `-DTREAT_WARNINGS_AS_ERRORS=OFF` [6008](https://github.com/Oneflow-Inc/oneflow/pull/6008)
- [**enhancement**][**build**][**xla**] xrt: fix all warnings [5915](https://github.com/Oneflow-Inc/oneflow/pull/5915)
- [**enhancement**][**build**] Prevent opencv compile failure with std 17 [5997](https://github.com/Oneflow-Inc/oneflow/pull/5997)
- [**enhancement**][**build**] Use bundled cub [5998](https://github.com/Oneflow-Inc/oneflow/pull/5998)
- [**enhancement**][**ci**][**build**] update clang tidy diff warnings-as-errors option [5989](https://github.com/Oneflow-Inc/oneflow/pull/5989)
- [**enhancement**][**build**] Update run_clang_tidy.py to set return code and add warning-as-errors [5977](https://github.com/Oneflow-Inc/oneflow/pull/5977)
- [**enhancement**][**build**] check: fix clang-tidy-diff commands [5972](https://github.com/Oneflow-Inc/oneflow/pull/5972)
- [**bug**][**build**] Suppress NVCC warning 177-D [6094](https://github.com/Oneflow-Inc/oneflow/pull/6094)

XLA enhancements:

- [**bug**][**xla**] Make the blob header memory aligned. [5286](https://github.com/Oneflow-Inc/oneflow/pull/5286)

System:

- [**enhancement**][**system**] Refactor Memory Zone [5072](https://github.com/Oneflow-Inc/oneflow/pull/5072)
- [**enhancement**][**system**] Add interface InferContext::OutputTensorDesc [5219](https://github.com/Oneflow-Inc/oneflow/pull/5219)
- [**bug**][**system**] Lazy construct functor to make sure that the operators has already been registered. [5225](https://github.com/Oneflow-Inc/oneflow/pull/5225)
- [**enhancement**][**system**] Refactor infer ctx output isdynamic [5220](https://github.com/Oneflow-Inc/oneflow/pull/5220)
- [**enhancement**][**system**] Refactor infer ctx input isdynamic [5211](https://github.com/Oneflow-Inc/oneflow/pull/5211)
- [**enhancement**][**system**] Wake up the heartbeat thread immediately [5081](https://github.com/Oneflow-Inc/oneflow/pull/5081)
- [**enhancement**][**system**] Fix xla test case fail [5203](https://github.com/Oneflow-Inc/oneflow/pull/5203)
- [**enhancement**][**system**] Add interface InferContext::InputDType [5153](https://github.com/Oneflow-Inc/oneflow/pull/5153)
- [**purge**][**system**] delete const_cast in Output [5196](https://github.com/Oneflow-Inc/oneflow/pull/5196)
- [**feature**][**system**] Add hwloc for topology detection [5291](https://github.com/Oneflow-Inc/oneflow/pull/5291)
- [**enhancement**][**system**] fix registry may segment [5336](https://github.com/Oneflow-Inc/oneflow/pull/5336)
- [**enhancement**][**system**] Use functional api instead of op_expr_helper::XXXOp. [5364](https://github.com/Oneflow-Inc/oneflow/pull/5364)
- [**enhancement**][**system**] move btob to op [5274](https://github.com/Oneflow-Inc/oneflow/pull/5274)
- [**documentation**][**system**] Add Latest News section in README [5361](https://github.com/Oneflow-Inc/oneflow/pull/5361)
- [**enhancement**][**bug**][**system**] fix dropout module: return directly if not training [5346](https://github.com/Oneflow-Inc/oneflow/pull/5346)
- [**bug**][**system**] add missing JUST [5357](https://github.com/Oneflow-Inc/oneflow/pull/5357)
- [**documentation**][**system**] Add more communication outlets on README [5359](https://github.com/Oneflow-Inc/oneflow/pull/5359)
- [**enhancement**][**feature**][**system**] CommNet dynamic register memory [5281](https://github.com/Oneflow-Inc/oneflow/pull/5281)
- [**enhancement**][**system**] Use symbol device [5341](https://github.com/Oneflow-Inc/oneflow/pull/5341)
- [**enhancement**][**system**] fix multithread bug in env [5283](https://github.com/Oneflow-Inc/oneflow/pull/5283)
- [**bug**][**system**][**api**] fix bug in cfg_replacement [5335](https://github.com/Oneflow-Inc/oneflow/pull/5335)
- [**bug**][**system**] Fix create log directory thread-unsafe [5326](https://github.com/Oneflow-Inc/oneflow/pull/5326)
- [**bug**][**system**] fix_bug_in_make_parallel [5328](https://github.com/Oneflow-Inc/oneflow/pull/5328)
- [**enhancement**][**system**][**cfg**] replace train_conf, job_conf using cfg::xx [5263](https://github.com/Oneflow-Inc/oneflow/pull/5263)
- [**enhancement**][**system**][**quantization**] support tensorrt in qat [5287](https://github.com/Oneflow-Inc/oneflow/pull/5287)
- [**enhancement**][**system**][**api**] Export functional apis for oneflow.experimental. [5313](https://github.com/Oneflow-Inc/oneflow/pull/5313)
- [**enhancement**][**system**] fix bug check between cfg enum and proto enum [5285](https://github.com/Oneflow-Inc/oneflow/pull/5285)
- [**enhancement**][**system**] replace CHECK_EQ using CHECK_EQ_OR_RETURN [5279](https://github.com/Oneflow-Inc/oneflow/pull/5279)
- [**enhancement**][**system**] Refactor SbpXXX to cfg::SbpXXX [5120](https://github.com/Oneflow-Inc/oneflow/pull/5120)
- [**enhancement**][**system**][**api**] add detach for LazyMirroredtensorImpl [5270](https://github.com/Oneflow-Inc/oneflow/pull/5270)
- [**enhancement**][**system**] shorten XXIsDynamic4ArgNameAndIndex to be xxIsDynamic [5265](https://github.com/Oneflow-Inc/oneflow/pull/5265)
- [**enhancement**][**system**][**cfg**] job_config to cfg [5235](https://github.com/Oneflow-Inc/oneflow/pull/5235)
- [**feature**][**system**] Multi-Client LogicalRun degenerate to PhysicalRun [5479](https://github.com/Oneflow-Inc/oneflow/pull/5479)
- [**enhancement**][**system**] fix ConstructOp without JUST [5480](https://github.com/Oneflow-Inc/oneflow/pull/5480)
- [**enhancement**][**system**] Output arg modifier return maybe part 1 [5451](https://github.com/Oneflow-Inc/oneflow/pull/5451)
- [**feature**][**system**][**interface**] Fea/nn graph/graph build ctx [5420](https://github.com/Oneflow-Inc/oneflow/pull/5420)
- [**enhancement**][**system**] Throw exception if check failed [5457](https://github.com/Oneflow-Inc/oneflow/pull/5457)
- [**feature**][**system**] multi client launch [5372](https://github.com/Oneflow-Inc/oneflow/pull/5372)
- [**enhancement**][**system**][**api**] Optimize reduce mean [5452](https://github.com/Oneflow-Inc/oneflow/pull/5452)
- [**enhancement**][**system**] export Tensor only to python [5440](https://github.com/Oneflow-Inc/oneflow/pull/5440)
- [**enhancement**][**system**] Output arg modifier return maybe part_0 [5447](https://github.com/Oneflow-Inc/oneflow/pull/5447)
- [**enhancement**][**system**] ThreadMgr support AddPlan [5450](https://github.com/Oneflow-Inc/oneflow/pull/5450)
- [**enhancement**][**system**] Refactor infer ctx input tensordesc [5226](https://github.com/Oneflow-Inc/oneflow/pull/5226)
- [**enhancement**][**system**][**api**] instruction builder return maybe [5442](https://github.com/Oneflow-Inc/oneflow/pull/5442)
- [**feature**][**system**][**interface**] MultiClientSessionContext [5421](https://github.com/Oneflow-Inc/oneflow/pull/5421)
- [**enhancement**][**feature**][**system**] add launcher, update multi client launch and exit [5414](https://github.com/Oneflow-Inc/oneflow/pull/5414)
- [**purge**][**system**][**refactor**] Remove IOConf [5419](https://github.com/Oneflow-Inc/oneflow/pull/5419)
- [**enhancement**][**system**] Dev refine generator [5426](https://github.com/Oneflow-Inc/oneflow/pull/5426)
- [**enhancement**][**system**] Support inplace operations [5204](https://github.com/Oneflow-Inc/oneflow/pull/5204)
- [**enhancement**][**system**][**refactor**] Dev refactor generator [5397](https://github.com/Oneflow-Inc/oneflow/pull/5397)
- [**enhancement**][**system**] Add new placement init func [5408](https://github.com/Oneflow-Inc/oneflow/pull/5408)
- [**enhancement**][**system**] NNGraphIf [5387](https://github.com/Oneflow-Inc/oneflow/pull/5387)
- [**enhancement**][**system**][**refactor**] Cast explicitily in unpack call to avoid confilt with Optional. [5380](https://github.com/Oneflow-Inc/oneflow/pull/5380)
- [**enhancement**][**system**][**interface**] [Random Generator] Part2: Migrate functional dropout [5378](https://github.com/Oneflow-Inc/oneflow/pull/5378)
- [**enhancement**][**system**] replace ForeignJobInstance using JobInstance [5374](https://github.com/Oneflow-Inc/oneflow/pull/5374)
- [**enhancement**][**system**][**refactor**] Speedup reshape module by 5x. [5381](https://github.com/Oneflow-Inc/oneflow/pull/5381)
- [**feature**][**system**][**interface**] [Random Generator] Part1: Dev random generator [5360](https://github.com/Oneflow-Inc/oneflow/pull/5360)
- [**enhancement**][**system**] Add ONEFLOW_STREAM_CUDA_EVENT_FLAG_BLOCKING_SYNC [5612](https://github.com/Oneflow-Inc/oneflow/pull/5612)
- [**enhancement**][**system**] [part2]Remove singleclient outdated api [5568](https://github.com/Oneflow-Inc/oneflow/pull/5568)
- [**feature**][**system**][**interface**] nn.Graph call and launch impl [5580](https://github.com/Oneflow-Inc/oneflow/pull/5580)
- [**enhancement**][**system**] remove outdated doctest api and "experimental_api" [5564](https://github.com/Oneflow-Inc/oneflow/pull/5564)
- [**feature**][**system**][**interface**] Register ForeignCallback and Watcher in Multi-Client [5591](https://github.com/Oneflow-Inc/oneflow/pull/5591)
- [**enhancement**][**system**] [Part-1]remove outdated api and files of multi-client on master branch [5556](https://github.com/Oneflow-Inc/oneflow/pull/5556)
- [**feature**][**system**][**interface**] LazyInterpret build LocalTensor if input is local [5582](https://github.com/Oneflow-Inc/oneflow/pull/5582)
- [**enhancement**][**system**] add job_pass MultiClientAutoSourceAndSinkTick [5507](https://github.com/Oneflow-Inc/oneflow/pull/5507)
- [**feature**][**system**] Fea/nn graph/optimizer [5533](https://github.com/Oneflow-Inc/oneflow/pull/5533)
- [**feature**][**system**][**interface**] New/CloseRuntimeBuffers and RunLazyJob impl [5571](https://github.com/Oneflow-Inc/oneflow/pull/5571)
- [**feature**][**system**][**refactor**][**interface**] NNGraph interface and implement for CompileAndRuntime [5558](https://github.com/Oneflow-Inc/oneflow/pull/5558)
- [**feature**][**system**] Fea/nn graph/forward graph [5516](https://github.com/Oneflow-Inc/oneflow/pull/5516)
- [**enhancement**][**system**] Lazy job stream type [5389](https://github.com/Oneflow-Inc/oneflow/pull/5389)
- [**enhancement**][**system**] Refactor single client autotick [5506](https://github.com/Oneflow-Inc/oneflow/pull/5506)
- [**enhancement**][**system**] replace underline using dot in single client [5547](https://github.com/Oneflow-Inc/oneflow/pull/5547)
- [**bug**][**system**] fix return type [5548](https://github.com/Oneflow-Inc/oneflow/pull/5548)
- [**feature**][**system**][**interface**] LazyInterpret for UserOpExpr [5544](https://github.com/Oneflow-Inc/oneflow/pull/5544)
- [**enhancement**][**system**] Add ProfilerStart/ProfilerStop API [5542](https://github.com/Oneflow-Inc/oneflow/pull/5542)
- [**feature**][**system**][**interface**] LazyInterpreter for FetchOutputOpExpr and set op parallel_distribution [5527](https://github.com/Oneflow-Inc/oneflow/pull/5527)
- [**enhancement**][**system**] Multi client push pull [5492](https://github.com/Oneflow-Inc/oneflow/pull/5492)
- [**enhancement**][**system**] registry_callback_fn return maybe [5456](https://github.com/Oneflow-Inc/oneflow/pull/5456)
- [**enhancement**][**system**] bw_gen_fn return maybe [5455](https://github.com/Oneflow-Inc/oneflow/pull/5455)
- [**enhancement**][**system**] gen_bw_fn return maybe [5454](https://github.com/Oneflow-Inc/oneflow/pull/5454)
- [**enhancement**][**system**] Compatible single client [5417](https://github.com/Oneflow-Inc/oneflow/pull/5417)
- [**feature**][**system**][**interface**] GlobalMultiClientEnv and refine EagerExecution [5523](https://github.com/Oneflow-Inc/oneflow/pull/5523)
- [**enhancement**][**system**] Job pass maybe system [5503](https://github.com/Oneflow-Inc/oneflow/pull/5503)
- [**enhancement**][**system**] Remove Plan::net_topo [5502](https://github.com/Oneflow-Inc/oneflow/pull/5502)
- [**feature**][**system**][**interface**] LazyInterpret for FeedVariableOpExpr [5490](https://github.com/Oneflow-Inc/oneflow/pull/5490)
- [**enhancement**][**system**] Input arg modifier return maybe [5453](https://github.com/Oneflow-Inc/oneflow/pull/5453)
- [**feature**][**system**][**interface**] Fea/nn graph/block scope [5498](https://github.com/Oneflow-Inc/oneflow/pull/5498)
- [**feature**][**system**] jit_fuse_cast_scale [5332](https://github.com/Oneflow-Inc/oneflow/pull/5332)
- [**enhancement**][**system**] Remove obsolete Profiler [5747](https://github.com/Oneflow-Inc/oneflow/pull/5747)
- [**enhancement**][**system**][**api**] Dev fix batch norm not stats [5733](https://github.com/Oneflow-Inc/oneflow/pull/5733)
- [**enhancement**][**system**] rename rpc_token to TransportToken [5735](https://github.com/Oneflow-Inc/oneflow/pull/5735)
- [**enhancement**][**system**][**api**] Refacotr maximum minimum py2cpp [5724](https://github.com/Oneflow-Inc/oneflow/pull/5724)
- [**enhancement**][**system**] Replace piece_id with comm_net_sequence_number [5731](https://github.com/Oneflow-Inc/oneflow/pull/5731)
- [**enhancement**][**system**] beautify stack frame [5686](https://github.com/Oneflow-Inc/oneflow/pull/5686)
- [**enhancement**][**system**] Add env ONEFLOW_KERNEL_DISABLE_BLOB_ACCESS_CHECKER [5728](https://github.com/Oneflow-Inc/oneflow/pull/5728)
- [**enhancement**][**system**] Add env ONEFLOW_THREAD_ENABLE_LOCAL_MESSAGE_QUEUE [5720](https://github.com/Oneflow-Inc/oneflow/pull/5720)
- [**enhancement**][**system**][**api**][**refactor**] Refactor functional sub, mul and div apis [5713](https://github.com/Oneflow-Inc/oneflow/pull/5713)
- [**feature**][**system**] ddp [5008](https://github.com/Oneflow-Inc/oneflow/pull/5008)
- [**enhancement**][**system**][**api**][**refactor**] Refactor functional matmul and add apis. [5697](https://github.com/Oneflow-Inc/oneflow/pull/5697)
- [**bug**][**system**] Fix ClearKV("plan") [5710](https://github.com/Oneflow-Inc/oneflow/pull/5710)
- [**enhancement**][**system**] Rename cpu to async cpu [5712](https://github.com/Oneflow-Inc/oneflow/pull/5712)
- [**enhancement**][**system**] Support tensor.to()/to_local() [5271](https://github.com/Oneflow-Inc/oneflow/pull/5271)
- [**feature**][**system**][**refactor**][**interface**] Multi-Runtime for multi nn.Graph [5683](https://github.com/Oneflow-Inc/oneflow/pull/5683)
- [**bug**][**system**][**refactor**] Add tag for Optional inplace constructor [5619](https://github.com/Oneflow-Inc/oneflow/pull/5619)
- [**enhancement**][**system**] Move Global<CommNet> to env scope [5670](https://github.com/Oneflow-Inc/oneflow/pull/5670)
- [**enhancement**][**system**] add JUST wrapper [5681](https://github.com/Oneflow-Inc/oneflow/pull/5681)
- [**enhancement**][**system**] New sync consistent meta info [5634](https://github.com/Oneflow-Inc/oneflow/pull/5634)
- [**enhancement**][**system**][**refactor**][**interface**] Refactor RuntimeCtx for multi-runtime [5664](https://github.com/Oneflow-Inc/oneflow/pull/5664)
- [**feature**][**system**][**interface**] Feat: memory shared between EagerTensor with VariableRegst [5649](https://github.com/Oneflow-Inc/oneflow/pull/5649)
- [**enhancement**][**system**] Use functional call directly instead of construct a module and then call-Add [5613](https://github.com/Oneflow-Inc/oneflow/pull/5613)
- [**enhancement**][**system**] disable eager_op consistent mode [5647](https://github.com/Oneflow-Inc/oneflow/pull/5647)
- [**enhancement**][**system**] add msg_penddin_list in ibverbs_qp to optimize qp_init_attr.cap.max_send_wr [5485](https://github.com/Oneflow-Inc/oneflow/pull/5485)
- [**enhancement**][**system**] IBVerbsCommNet add knobs [5626](https://github.com/Oneflow-Inc/oneflow/pull/5626)
- [**enhancement**][**system**] Prune python tensor [5596](https://github.com/Oneflow-Inc/oneflow/pull/5596)
- [**feature**][**system**][**interface**] Feat: LazyInterpret infer op / tensor ParallelDescScope [5625](https://github.com/Oneflow-Inc/oneflow/pull/5625)
- [**enhancement**][**system**] Replace src tick with with wait and send ids [5603](https://github.com/Oneflow-Inc/oneflow/pull/5603)
- [**enhancement**][**system**] Support symbol placement type in functional. [5627](https://github.com/Oneflow-Inc/oneflow/pull/5627)
- [**enhancement**][**system**][**api**][**refactor**][**interface**] Dev advanced indexing [5559](https://github.com/Oneflow-Inc/oneflow/pull/5559)
- [**enhancement**][**system**] Optimize maybe. [5839](https://github.com/Oneflow-Inc/oneflow/pull/5839)
- [**enhancement**][**system**] Decorator 4 disable recursive boxing call [5796](https://github.com/Oneflow-Inc/oneflow/pull/5796)
- [**enhancement**][**system**] add_eager_boxing_and_op_interpreter_dispatch_error_info [5819](https://github.com/Oneflow-Inc/oneflow/pull/5819)
- [**enhancement**][**system**] Kernel CUDA Graphs Support [5725](https://github.com/Oneflow-Inc/oneflow/pull/5725)
- [**bug**][**system**] Fix placement print bug [5853](https://github.com/Oneflow-Inc/oneflow/pull/5853)
- [**bug**][**system**] when error msg formatting fails, return error->DebugString [5844](https://github.com/Oneflow-Inc/oneflow/pull/5844)
- [**enhancement**][**system**][**refactor**] Rename variables named `*parallel_distribution*` to `*nd_sbp*` (1) [5815](https://github.com/Oneflow-Inc/oneflow/pull/5815)
- [**feature**][**system**][**interface**] Support Free EagerTensor caught in nn.Graph build [5777](https://github.com/Oneflow-Inc/oneflow/pull/5777)
- [**enhancement**][**system**] Reuse CUDA event / Refine BnInOp2Blob / Refine channel [5837](https://github.com/Oneflow-Inc/oneflow/pull/5837)
- [**enhancement**][**system**][**serving**] fix bug in AddInputOutputOpsPass: check existence of key in HashMap(inferface_lbi2scope_sym_id) [5653](https://github.com/Oneflow-Inc/oneflow/pull/5653)
- [**enhancement**][**system**][**api**] unpack_call: impl new `unpack_call_dispatcher` for better performance [5820](https://github.com/Oneflow-Inc/oneflow/pull/5820)
- [**feature**][**system**] Feat consistent tensor python constructor [5812](https://github.com/Oneflow-Inc/oneflow/pull/5812)
- [**feature**][**system**] Support 0shape tensor [5620](https://github.com/Oneflow-Inc/oneflow/pull/5620)
- [**documentation**][**system**] fix launcher description [5770](https://github.com/Oneflow-Inc/oneflow/pull/5770)
- [**feature**][**system**][**interface**] Multi-nn.Graph memory reuse by Chunk manager [5658](https://github.com/Oneflow-Inc/oneflow/pull/5658)
- [**bug**][**system**] Fix naive b2p error [5806](https://github.com/Oneflow-Inc/oneflow/pull/5806)
- [**enhancement**][**system**] set created generator with default rng seed [5801](https://github.com/Oneflow-Inc/oneflow/pull/5801)
- [**enhancement**][**system**] enhance_local_to_consistent [5761](https://github.com/Oneflow-Inc/oneflow/pull/5761)
- [**feature**][**system**] add flow.randn [5736](https://github.com/Oneflow-Inc/oneflow/pull/5736)
- [**enhancement**][**system**] Refactor hierarchical parallel cast autograd [5764](https://github.com/Oneflow-Inc/oneflow/pull/5764)
- [**enhancement**][**system**] Collective boxing executor add_plan delete_plan [5495](https://github.com/Oneflow-Inc/oneflow/pull/5495)
- [**enhancement**][**system**] Fix throw abort [5795](https://github.com/Oneflow-Inc/oneflow/pull/5795)
- [**enhancement**][**system**] DECORATE [5794](https://github.com/Oneflow-Inc/oneflow/pull/5794)
- [**enhancement**][**system**] Inferface eager boxing [5682](https://github.com/Oneflow-Inc/oneflow/pull/5682)
- [**enhancement**][**system**] extract_consistent_to_consistent_op_expr [5870](https://github.com/Oneflow-Inc/oneflow/pull/5870)
- [**enhancement**][**system**] disable backward pass consistent tensor meta check. [5871](https://github.com/Oneflow-Inc/oneflow/pull/5871)
- [**enhancement**][**system**] Add CudaStreamIndexGenerator::GenerateNamedStreamIndex [5940](https://github.com/Oneflow-Inc/oneflow/pull/5940)
- [**bug**][**system**] Only query PCI bus id when CUDA version >= 11 [5937](https://github.com/Oneflow-Inc/oneflow/pull/5937)
- [**enhancement**][**system**] maybe: add `JUST_MSG` and `CHECK_JUST_MSG` [5904](https://github.com/Oneflow-Inc/oneflow/pull/5904)
- [**bug**][**system**] Fix bug scalar [5950](https://github.com/Oneflow-Inc/oneflow/pull/5950)
- [**enhancement**][**system**] framework: fix rvalue reference warnings [5948](https://github.com/Oneflow-Inc/oneflow/pull/5948)
- [**purge**][**system**] Remove CudaWorkType [5942](https://github.com/Oneflow-Inc/oneflow/pull/5942)
- [**enhancement**][**system**] refactor_symbol [5941](https://github.com/Oneflow-Inc/oneflow/pull/5941)
- [**bug**][**system**] consistent_tensor_infer_cache: fix memory leak [5938](https://github.com/Oneflow-Inc/oneflow/pull/5938)
- [**feature**][**system**] support to print gpu [5936](https://github.com/Oneflow-Inc/oneflow/pull/5936)
- [**enhancement**][**system**] Bugfix static check [5935](https://github.com/Oneflow-Inc/oneflow/pull/5935)
- [**bug**][**system**] fix nccl_version log [5934](https://github.com/Oneflow-Inc/oneflow/pull/5934)
- [**bug**][**system**] Fix bug of multi-GPU train nn.Graph extra mem cost in rank 0 [5930](https://github.com/Oneflow-Inc/oneflow/pull/5930)
- [**enhancement**][**system**] Only gradient acc be scheduled in parallel. [5926](https://github.com/Oneflow-Inc/oneflow/pull/5926)
- [**enhancement**][**bug**][**system**] fix_ddp_bug_on_8_process [5929](https://github.com/Oneflow-Inc/oneflow/pull/5929)
- [**enhancement**][**system**] Fix bug error msg format [5866](https://github.com/Oneflow-Inc/oneflow/pull/5866)
- [**feature**][**system**] print consistent tensor data [5902](https://github.com/Oneflow-Inc/oneflow/pull/5902)
- [**bug**][**system**] Move parse env to the constructor [5922](https://github.com/Oneflow-Inc/oneflow/pull/5922)
- [**enhancement**][**system**] Remove GlobalWorkStreamId/GlobalThrdId [5917](https://github.com/Oneflow-Inc/oneflow/pull/5917)
- [**bug**][**system**] shared_or_scalar: fix alias warnings [5916](https://github.com/Oneflow-Inc/oneflow/pull/5916)
- [**purge**][**system**] Remove CompActor [5919](https://github.com/Oneflow-Inc/oneflow/pull/5919)
- [**enhancement**][**system**] Use symbol dtype [5641](https://github.com/Oneflow-Inc/oneflow/pull/5641)
- [**enhancement**][**feature**][**system**] Control Graph / Session / Env's python c++ object destruction [5845](https://github.com/Oneflow-Inc/oneflow/pull/5845)
- [**enhancement**][**bug**][**system**] Sync access and assign indexing tensor. [5907](https://github.com/Oneflow-Inc/oneflow/pull/5907)
- [**enhancement**][**system**][**api**][**refactor**] Dev consistent arange [5883](https://github.com/Oneflow-Inc/oneflow/pull/5883)
- [**enhancement**][**system**] Lazy interpreter for new ConsistentToConsistentOpExpr [5903](https://github.com/Oneflow-Inc/oneflow/pull/5903)
- [**bug**][**system**] Fix BUG of LazyInterpret FreeEagerTensor memory shared with regst [5891](https://github.com/Oneflow-Inc/oneflow/pull/5891)
- [**bug**][**system**] fix typo in `raise RuntimeError` [5890](https://github.com/Oneflow-Inc/oneflow/pull/5890)
- [**enhancement**][**system**][**refactor**] Rename the `ParallelDistribution` class to `NdSbp` [5814](https://github.com/Oneflow-Inc/oneflow/pull/5814)
- [**feature**][**system**] add flow.rand [5722](https://github.com/Oneflow-Inc/oneflow/pull/5722)
- [**feature**][**system**] Lazy Interpret support infer default device cpu [5880](https://github.com/Oneflow-Inc/oneflow/pull/5880)
- [**enhancement**][**system**] Tensor str [5783](https://github.com/Oneflow-Inc/oneflow/pull/5783)
- [**feature**][**system**][**interface**] Lazy to_consistent [5774](https://github.com/Oneflow-Inc/oneflow/pull/5774)
- [**enhancement**][**system**] wait vm empty before exiting [5860](https://github.com/Oneflow-Inc/oneflow/pull/5860)
- [**enhancement**][**system**] Eager boxing n to 1 [5949](https://github.com/Oneflow-Inc/oneflow/pull/5949)
- [**enhancement**][**system**] add kernel observer [6052](https://github.com/Oneflow-Inc/oneflow/pull/6052)
- [**enhancement**][**ci**][**system**] Optimize ddp broadcast and add speed/memory test in ci [6044](https://github.com/Oneflow-Inc/oneflow/pull/6044)
- [**enhancement**][**system**] add var to control only print warning once when blocked [6045](https://github.com/Oneflow-Inc/oneflow/pull/6045)
- [**enhancement**][**system**][**refactor**] Rewrite pow and logical functional apis [6032](https://github.com/Oneflow-Inc/oneflow/pull/6032)
- [**enhancement**][**system**] Token seq id [5964](https://github.com/Oneflow-Inc/oneflow/pull/5964)
- [**enhancement**][**documentation**][**system**] Remove python function wrapper. [6012](https://github.com/Oneflow-Inc/oneflow/pull/6012)
- [**feature**][**system**] Add timeout and loc for blocking calls [6007](https://github.com/Oneflow-Inc/oneflow/pull/6007)
- [**enhancement**][**system**] Eager boxing 1 to n [5943](https://github.com/Oneflow-Inc/oneflow/pull/5943)
- [**enhancement**][**system**] Boxing expr [6015](https://github.com/Oneflow-Inc/oneflow/pull/6015)
- [**enhancement**][**system**] new_X_to_B [5987](https://github.com/Oneflow-Inc/oneflow/pull/5987)
- [**enhancement**][**system**] Add unimplemented return information [5952](https://github.com/Oneflow-Inc/oneflow/pull/5952)
- [**enhancement**][**system**] Revert "Faster decorator" [6006](https://github.com/Oneflow-Inc/oneflow/pull/6006)
- [**enhancement**][**system**] Throw exception if using advanced indexing for tensor setitem [6001](https://github.com/Oneflow-Inc/oneflow/pull/6001)
- [**enhancement**][**system**] Support eager boxing sm 2 sn [5869](https://github.com/Oneflow-Inc/oneflow/pull/5869)
- [**enhancement**][**system**] Move framework/local_dep_object.\* to the eager directory [5988](https://github.com/Oneflow-Inc/oneflow/pull/5988)
- [**enhancement**][**system**] Fix builtin op arg tuple. [5464](https://github.com/Oneflow-Inc/oneflow/pull/5464)
- [**feature**][**system**][**refactor**] Dev functional multiple signatures [5982](https://github.com/Oneflow-Inc/oneflow/pull/5982)
- [**enhancement**][**system**] Faster decorator [5996](https://github.com/Oneflow-Inc/oneflow/pull/5996)
- [**enhancement**][**system**] Placed nd sbp [5995](https://github.com/Oneflow-Inc/oneflow/pull/5995)
- [**feature**][**system**] Support asymmetric input/output/variable tensors in nn.Graph [5983](https://github.com/Oneflow-Inc/oneflow/pull/5983)
- [**enhancement**][**system**] LightActor [5868](https://github.com/Oneflow-Inc/oneflow/pull/5868)
- [**bug**][**system**] Prevent running oneflow in forked subprocess [5976](https://github.com/Oneflow-Inc/oneflow/pull/5976)
- [**bug**][**system**] common/error: fix build error in mac os [5971](https://github.com/Oneflow-Inc/oneflow/pull/5971)
- [**bug**][**system**] fix_bug_test_tensor_str [5958](https://github.com/Oneflow-Inc/oneflow/pull/5958)
- [**enhancement**][**system**] Refine StreamContext [6191](https://github.com/Oneflow-Inc/oneflow/pull/6191)
- [**enhancement**][**system**] container_util: fix VectorAt, remove useless MutMapAt [6172](https://github.com/Oneflow-Inc/oneflow/pull/6172)
- [**enhancement**][**system**] Typesafe KernelState [6198](https://github.com/Oneflow-Inc/oneflow/pull/6198)
- [**enhancement**][**system**] Primitive based copy task node [6195](https://github.com/Oneflow-Inc/oneflow/pull/6195)
- [**feature**][**system**][**interface**] Lazy support Scalar [6181](https://github.com/Oneflow-Inc/oneflow/pull/6181)
- [**enhancement**][**system**] Disable implicit boxing when parallel num eq one [6188](https://github.com/Oneflow-Inc/oneflow/pull/6188)
- [**enhancement**][**system**] Primitive [6183](https://github.com/Oneflow-Inc/oneflow/pull/6183)
- [**enhancement**][**system**] Remove IDMgr::GetGpuPhyIdFromThrdId/IDMgr::GetDeviceTypeFromThrdId [6169](https://github.com/Oneflow-Inc/oneflow/pull/6169)
- [**enhancement**][**system**] remove op_expr_helper inside gradient_funcs [6057](https://github.com/Oneflow-Inc/oneflow/pull/6057)
- [**feature**][**system**][**api**] Add tensor yaml, support export tensor functional api. [6099](https://github.com/Oneflow-Inc/oneflow/pull/6099)
- [**feature**][**system**] Plan memory log [6151](https://github.com/Oneflow-Inc/oneflow/pull/6151)
- [**feature**][**system**] Add dtype bfloat16 [5304](https://github.com/Oneflow-Inc/oneflow/pull/5304)
- [**enhancement**][**system**] StreamContext [6129](https://github.com/Oneflow-Inc/oneflow/pull/6129)
- [**bug**][**system**] Fix wrong inplace acc grad [6146](https://github.com/Oneflow-Inc/oneflow/pull/6146)
- [**enhancement**][**system**] UserKernel remove job_desc [6144](https://github.com/Oneflow-Inc/oneflow/pull/6144)
- [**enhancement**][**system**][**api**] Fea/graph/add outputs buffer to enable pipeline [6126](https://github.com/Oneflow-Inc/oneflow/pull/6126)
- [**enhancement**][**system**] not fuse request for nccl 2.10.3 [6136](https://github.com/Oneflow-Inc/oneflow/pull/6136)
- [**bug**][**system**] NewUniqueId thread safe [6141](https://github.com/Oneflow-Inc/oneflow/pull/6141)
- [**enhancement**][**system**] XRT remove job_desc [6139](https://github.com/Oneflow-Inc/oneflow/pull/6139)
- [**enhancement**][**system**] SystemOpFillJobNamePass [6138](https://github.com/Oneflow-Inc/oneflow/pull/6138)
- [**enhancement**][**system**] mv_boxing_folder_to_core [6140](https://github.com/Oneflow-Inc/oneflow/pull/6140)
- [**enhancement**][**system**] Refactor boxing interpreter to boxing expr [6134](https://github.com/Oneflow-Inc/oneflow/pull/6134)
- [**enhancement**][**system**] Eager boxing one to one [6048](https://github.com/Oneflow-Inc/oneflow/pull/6048)
- [**enhancement**][**system**] Vm cpu efficiency [6110](https://github.com/Oneflow-Inc/oneflow/pull/6110)
- [**enhancement**][**system**] Naive generic boxing [6116](https://github.com/Oneflow-Inc/oneflow/pull/6116)
- [**feature**][**system**] send/recv [5992](https://github.com/Oneflow-Inc/oneflow/pull/5992)
- [**enhancement**][**system**] disable_print_stack_in_tensor_numpy [6123](https://github.com/Oneflow-Inc/oneflow/pull/6123)
- [**feature**][**system**] add all_reduce by to_consistent [5963](https://github.com/Oneflow-Inc/oneflow/pull/5963)
- [**enhancement**][**system**] KernelContext [6084](https://github.com/Oneflow-Inc/oneflow/pull/6084)
- [**enhancement**][**bug**][**system**] Fix sync nccl and async nccl deadlock [6071](https://github.com/Oneflow-Inc/oneflow/pull/6071)
- [**bug**][**system**][**refactor**] Refactor to local [6098](https://github.com/Oneflow-Inc/oneflow/pull/6098)
- [**enhancement**][**system**] Replace xor with hash combine (part 1) [6078](https://github.com/Oneflow-Inc/oneflow/pull/6078)
- [**enhancement**][**system**] Optimize error message [6073](https://github.com/Oneflow-Inc/oneflow/pull/6073)
- [**enhancement**][**system**] Rename Error::xx to Error::xxError [6049](https://github.com/Oneflow-Inc/oneflow/pull/6049)
- [**enhancement**][**system**] send formatted msg to glog [5999](https://github.com/Oneflow-Inc/oneflow/pull/5999)
- [**feature**][**bottleneck**][**bug**][**system**][**interface**] [Feat.] NNGraph new eager tensor for new variable created in JobPass [6091](https://github.com/Oneflow-Inc/oneflow/pull/6091)
- [**bug**][**system**] Fix bug of multi-GPU eager copy D2H extra mem cost in rank 0 [6092](https://github.com/Oneflow-Inc/oneflow/pull/6092)
- [**enhancement**][**system**][**api**] Rename module flow.F to flow.\_C [6053](https://github.com/Oneflow-Inc/oneflow/pull/6053)
- [**feature**][**system**][**interface**] [Feat.] Eager consistent OFRecordReader [6089](https://github.com/Oneflow-Inc/oneflow/pull/6089)
- [**enhancement**][**system**][**api**] Dev fix and align interface [6075](https://github.com/Oneflow-Inc/oneflow/pull/6075)
- [**feature**][**bottleneck**][**bug**][**system**][**interface**] NNGraph input/output valid by register tensors [6240](https://github.com/Oneflow-Inc/oneflow/pull/6240)
- [**bug**][**system**][**interface**] Fix bug of Multi-Client src tick output order [6221](https://github.com/Oneflow-Inc/oneflow/pull/6221)
- [**enhancement**][**bug**][**system**] Add cast primitive [6234](https://github.com/Oneflow-Inc/oneflow/pull/6234)
- [**feature**][**bottleneck**][**system**][**interface**] Auto FixPipelineStageIdPass [6204](https://github.com/Oneflow-Inc/oneflow/pull/6204)
- [**enhancement**][**system**] move scalar to oneflow namespace. [6235](https://github.com/Oneflow-Inc/oneflow/pull/6235)
- [**enhancement**][**system**] UserKernel init CUDA Graphs with state [6230](https://github.com/Oneflow-Inc/oneflow/pull/6230)
- [**feature**][**system**] Comm broadcast [6213](https://github.com/Oneflow-Inc/oneflow/pull/6213)
- [**enhancement**][**system**][**refactor**] Rename op*name* to op*type_name* in AutogradEngine [6154](https://github.com/Oneflow-Inc/oneflow/pull/6154)
- [**enhancement**][**system**] Add memset primitive [6218](https://github.com/Oneflow-Inc/oneflow/pull/6218)
- [**enhancement**][**system**] Add StreamContext::device_type()/DeviceCtx::device_type() [6217](https://github.com/Oneflow-Inc/oneflow/pull/6217)
- [**feature**][**system**] add all_gather and fix bug of multi rank doctest [6189](https://github.com/Oneflow-Inc/oneflow/pull/6189)
- [**feature**][**system**][**interface**] [Feat.] Lazy interpreter skip hierarchical_parallel_cast [6208](https://github.com/Oneflow-Inc/oneflow/pull/6208)
- [**purge**][**system**] Cleanup KernelUtil [6212](https://github.com/Oneflow-Inc/oneflow/pull/6212)
- [**enhancement**][**system**] StreamContextAdapter [6205](https://github.com/Oneflow-Inc/oneflow/pull/6205)
- [**enhancement**][**system**] Dev eliminate gcc warnings [6199](https://github.com/Oneflow-Inc/oneflow/pull/6199)
- [**feature**][**bottleneck**][**system**][**interface**] [Feat.] nn.Graph support grad acc with input/output tensor [6155](https://github.com/Oneflow-Inc/oneflow/pull/6155)
- [**enhancement**][**system**] Cpu symetric s to s [6153](https://github.com/Oneflow-Inc/oneflow/pull/6153)
- [**enhancement**][**system**][**upload-core**] Op expr infer tensor meta [5064](https://github.com/Oneflow-Inc/oneflow/pull/5064)
- [**enhancement**][**system**] Infer consistent tensor meta [5362](https://github.com/Oneflow-Inc/oneflow/pull/5362)

CI enhancements:

- [**bug**][**ci**][**api**][**interface**] Refine module test [5232](https://github.com/Oneflow-Inc/oneflow/pull/5232)
- [**enhancement**][**ci**] Add Simple CI, runs CPU-only on GitHub hosted servers [5207](https://github.com/Oneflow-Inc/oneflow/pull/5207)
- [**enhancement**][**ci**] Run exe test in CPU-only [5202](https://github.com/Oneflow-Inc/oneflow/pull/5202)
- [**enhancement**][**ci**] Cancel all workflow runs but the latest [5206](https://github.com/Oneflow-Inc/oneflow/pull/5206)
- [**enhancement**][**ci**] Fix master not running Simple CI [5368](https://github.com/Oneflow-Inc/oneflow/pull/5368)
- [**enhancement**][**ci**] Refine Simple CI and Clang analysis [5367](https://github.com/Oneflow-Inc/oneflow/pull/5367)
- [**enhancement**][**feature**][**bug**][**ci**][**documentation**][**interface**] Fix upsample bilinear bug [5363](https://github.com/Oneflow-Inc/oneflow/pull/5363)
- [**enhancement**][**ci**] Build nightly for py39 [5318](https://github.com/Oneflow-Inc/oneflow/pull/5318)
- [**enhancement**][**ci**] Try distributed run for 3 times to prevent failure [5305](https://github.com/Oneflow-Inc/oneflow/pull/5305)
- [**enhancement**][**ci**] Upload Simple CI logs to cloud [5268](https://github.com/Oneflow-Inc/oneflow/pull/5268)
- [**enhancement**][**ci**] Remove cpu_op_eager and cuda_op_eager [5470](https://github.com/Oneflow-Inc/oneflow/pull/5470)
- [**bug**][**ci**] fix segfault in clang plugin [5437](https://github.com/Oneflow-Inc/oneflow/pull/5437)
- [**enhancement**][**ci**] Refine Simple CI error output [5435](https://github.com/Oneflow-Inc/oneflow/pull/5435)
- [**enhancement**][**ci**] Add conda env to Simple CI [5385](https://github.com/Oneflow-Inc/oneflow/pull/5385)
- [**enhancement**][**ci**] Fix clang plugin core file not found [5390](https://github.com/Oneflow-Inc/oneflow/pull/5390)
- [**bug**][**ci**] upload core when build with clang plugin [5384](https://github.com/Oneflow-Inc/oneflow/pull/5384)
- [**bug**][**ci**] clang plugin skip more files [5373](https://github.com/Oneflow-Inc/oneflow/pull/5373)
- [**enhancement**][**ci**] Use gh-action-scheduler-v2 [5370](https://github.com/Oneflow-Inc/oneflow/pull/5370)
- [**enhancement**][**ci**] relax speed threshold [5569](https://github.com/Oneflow-Inc/oneflow/pull/5569)
- [**bug**][**ci**] Fix wrong test path under compatible [5567](https://github.com/Oneflow-Inc/oneflow/pull/5567)
- [**enhancement**][**ci**][**need-simple-ci**] Prevent upload logs automatically [5560](https://github.com/Oneflow-Inc/oneflow/pull/5560)
- [**enhancement**][**ci**][**interface**] Add `nn.AdaptiveAvgPool1d` and `nn.AdaptiveAvgPool3d` [5445](https://github.com/Oneflow-Inc/oneflow/pull/5445)
- [**feature**][**ci**] add speed test in ci [5496](https://github.com/Oneflow-Inc/oneflow/pull/5496)
- [**enhancement**][**ci**] Reduce usage of Simple CI [5546](https://github.com/Oneflow-Inc/oneflow/pull/5546)
- [**feature**][**bug**][**ci**][**api**] Restruct upsample module [5524](https://github.com/Oneflow-Inc/oneflow/pull/5524)
- [**feature**][**ci**] multi client launcher test [5488](https://github.com/Oneflow-Inc/oneflow/pull/5488)
- [**enhancement**][**ci**] Remove automerge if cuda_new_interface failed [5519](https://github.com/Oneflow-Inc/oneflow/pull/5519)
- [**enhancement**][**ci**] Prevent adding subdir in python/test [5514](https://github.com/Oneflow-Inc/oneflow/pull/5514)
- [**enhancement**][**ci**] piprepo->pipindex [5517](https://github.com/Oneflow-Inc/oneflow/pull/5517)
- [**enhancement**][**ci**] add dynamic_loss_scale in ci tests [5337](https://github.com/Oneflow-Inc/oneflow/pull/5337)
- [**enhancement**][**ci**] Add timeout for wait_gpu_slot [5497](https://github.com/Oneflow-Inc/oneflow/pull/5497)
- [**enhancement**][**feature**][**ci**] new static check based on clang-tidy [5476](https://github.com/Oneflow-Inc/oneflow/pull/5476)
- [**enhancement**][**ci**] Fix url not downloadable in some browers [5701](https://github.com/Oneflow-Inc/oneflow/pull/5701)
- [**feature**][**ci**] multi client multi machine test [5685](https://github.com/Oneflow-Inc/oneflow/pull/5685)
- [**enhancement**][**ci**] Add cpu new interface CI [5639](https://github.com/Oneflow-Inc/oneflow/pull/5639)
- [**enhancement**][**ci**][**need-simple-ci**] Mv clangtidy to simple ci [5667](https://github.com/Oneflow-Inc/oneflow/pull/5667)
- [**enhancement**][**ci**][**need-simple-ci**] use clang tidy appimage in ci [5841](https://github.com/Oneflow-Inc/oneflow/pull/5841)
- [**enhancement**][**ci**] Use gcc 7 in release to prevent error [5840](https://github.com/Oneflow-Inc/oneflow/pull/5840)
- [**enhancement**][**ci**] bn tol 1e-4 => 1e-3 [5811](https://github.com/Oneflow-Inc/oneflow/pull/5811)
- [**enhancement**][**ci**] fix distributed run on built dir [5810](https://github.com/Oneflow-Inc/oneflow/pull/5810)
- [**enhancement**][**ci**] fix third party mirror check_sum [5802](https://github.com/Oneflow-Inc/oneflow/pull/5802)
- [**ci**][**documentation**] find more accurately which files need to be doctested [5782](https://github.com/Oneflow-Inc/oneflow/pull/5782)
- [**enhancement**][**ci**] Print stack unconditionally [5779](https://github.com/Oneflow-Inc/oneflow/pull/5779)
- [**enhancement**][**ci**][**need-simple-ci**] Enable more checkers for clang-tidy in CI [5738](https://github.com/Oneflow-Inc/oneflow/pull/5738)
- [**enhancement**][**ci**] CI: add clang-tidy check to test.yaml [5920](https://github.com/Oneflow-Inc/oneflow/pull/5920)
- [**ci**][**documentation**] fix docstring in oneflow.nn.functional namespace [5807](https://github.com/Oneflow-Inc/oneflow/pull/5807)
- [**enhancement**][**ci**] disable TREAT_WARNINGS_AS_ERRORS in Release CI [5886](https://github.com/Oneflow-Inc/oneflow/pull/5886)
- [**enhancement**][**ci**] Skip ci jobs by git diff [5863](https://github.com/Oneflow-Inc/oneflow/pull/5863)
- [**bug**][**ci**] quick fix 5978 [6030](https://github.com/Oneflow-Inc/oneflow/pull/6030)
- [**enhancement**][**bug**][**ci**] fix clang tidy diff options and file format [5990](https://github.com/Oneflow-Inc/oneflow/pull/5990)
- [**enhancement**][**ci**] add flow.relu [5847](https://github.com/Oneflow-Inc/oneflow/pull/5847)
- [**enhancement**][**ci**] equal => allclose [6164](https://github.com/Oneflow-Inc/oneflow/pull/6164)
- [**bug**][**ci**][**need-simple-ci**] CI: fix clang tidy checks in simple ci [6161](https://github.com/Oneflow-Inc/oneflow/pull/6161)
- [**enhancement**][**bug**][**ci**][**documentation**][**api**] add interpolate and layer_norm docs [6157](https://github.com/Oneflow-Inc/oneflow/pull/6157)
- [**bug**][**ci**] update speed test [6113](https://github.com/Oneflow-Inc/oneflow/pull/6113)
- [**enhancement**][**bug**][**ci**][**documentation**][**api**] speed import oneflow [6107](https://github.com/Oneflow-Inc/oneflow/pull/6107)
- [**bug**][**ci**] Also try install dev deps for CODEGEN_PYTHON_EXECUTABLE [6115](https://github.com/Oneflow-Inc/oneflow/pull/6115)
- [**bug**][**ci**][**need-simple-ci**] set gtest_CMAKE_DEBUG_POSTFIX "d" [6085](https://github.com/Oneflow-Inc/oneflow/pull/6085)
- [**enhancement**][**ci**] add cache init file for clang and CI build with clang [6062](https://github.com/Oneflow-Inc/oneflow/pull/6062)
- [**enhancement**][**ci**] add emoji in speed test output, make it continue-on-error [6214](https://github.com/Oneflow-Inc/oneflow/pull/6214)

Test enhancements:

- [**bug**][**test**][**interface**] Fix acos ci bug [5217](https://github.com/Oneflow-Inc/oneflow/pull/5217)
- [**feature**][**test**] implement automated test [5321](https://github.com/Oneflow-Inc/oneflow/pull/5321)
- [**enhancement**][**test**] move generator test into ops folder to accelerate tests [5472](https://github.com/Oneflow-Inc/oneflow/pull/5472)
- [**feature**][**test**][**api**] Add autotest part2 [5467](https://github.com/Oneflow-Inc/oneflow/pull/5467)
- [**enhancement**][**test**][**api**][**interface**] Add some tests with the new framework for auto testing [5561](https://github.com/Oneflow-Inc/oneflow/pull/5561)
- [**bug**][**test**] fix test error when do multi case test on graph [5590](https://github.com/Oneflow-Inc/oneflow/pull/5590)
- [**enhancement**][**test**] Refine module test using auto test by yaochi [5484](https://github.com/Oneflow-Inc/oneflow/pull/5484)
- [**enhancement**][**test**] Add autotest for BatchNorm2d [5734](https://github.com/Oneflow-Inc/oneflow/pull/5734)
- [**enhancement**][**test**] RTH_update_op_test [5823](https://github.com/Oneflow-Inc/oneflow/pull/5823)
- [**enhancement**][**test**] dev adamw graph config [5745](https://github.com/Oneflow-Inc/oneflow/pull/5745)
- [**feature**][**test**][**api**][**interface**] Add new autotest [5562](https://github.com/Oneflow-Inc/oneflow/pull/5562)
- [**bug**][**test**] restore test of alexnet graph [5798](https://github.com/Oneflow-Inc/oneflow/pull/5798)
- [**enhancement**][**test**][**interface**] add zhangshen op-test [5600](https://github.com/Oneflow-Inc/oneflow/pull/5600)
- [**feature**][**bug**][**tooling**][**test**][**interface**] Record autotest wrong code [5923](https://github.com/Oneflow-Inc/oneflow/pull/5923)
- [**enhancement**][**feature**][**test**][**api**] add randint [5718](https://github.com/Oneflow-Inc/oneflow/pull/5718)
- [**bug**][**test**] fix multi machine test [5984](https://github.com/Oneflow-Inc/oneflow/pull/5984)
- [**enhancement**][**test**][**interface**] some op test [6095](https://github.com/Oneflow-Inc/oneflow/pull/6095)

Tooling enhancements:

- [**bug**][**tooling**] user/summary: fix memory leak in `FillImageInSummary` [5742](https://github.com/Oneflow-Inc/oneflow/pull/5742)
- [**enhancement**][**tooling**][**cfg**] cfg: add move assignment operator for performance [5962](https://github.com/Oneflow-Inc/oneflow/pull/5962)
- [**enhancement**][**tooling**][**api**][**refactor**] refactor_all_device_placement_api [6080](https://github.com/Oneflow-Inc/oneflow/pull/6080)

Highlights

- First class support for eager execution. The deprecated APIs are moved to `oneflow.compatible.single_client`
- Drop-in replacement of `import torch` for existing Pytorch projects. You could test it by inter-changing `import oneflow as torch` and `import torch as flow`.
- [nn.Module](https://docs.oneflow.org/master/basics/04_build_network.html#module) for eager execution
- [nn.Graph](https://docs.oneflow.org/master/basics/08_nn_graph.html) for lazy execution
- [DDP](https://oneflow.readthedocs.io/en/master/nn.html#oneflow.nn.parallel.DistributedDataParallel) for data parallel

A sneak peek of the new API

Here is a minimum example showcasing how to incorporate a `nn.Module` in a `nn.Graph` and have it run in lazy mode.

python
class NeuralGraph(flow.nn.Graph):
def __init__(self, ...):
super().__init__()
self.model = model model is a nn.Module instance

def build(self, x):
y_pred = self.model(x)
return y_pred

graph = NeuralGraph() to create a nn.Graph instance
y_pred = graph(x) to run the created nn.Graph


New in Python API

- [**feature**][**eager**][**op**][**test**][**python**][**interface**] Add test for convtranspose2d [5239](https://github.com/Oneflow-Inc/oneflow/pull/5239)
- [**enhancement**][**python**][**interface**] Add GroupNorm [5175](https://github.com/Oneflow-Inc/oneflow/pull/5175)
- [**enhancement**][**eager**][**python**][**interface**] [Add] avgpool1d avgpool3d [5165](https://github.com/Oneflow-Inc/oneflow/pull/5165)
- [**feature**][**eager**][**op**][**python**][**interface**] Add deconv cpu impl [5224](https://github.com/Oneflow-Inc/oneflow/pull/5224)
- [**bug**][**eager**][**api**][**python**][**interface**] Fix acosh bug [5221](https://github.com/Oneflow-Inc/oneflow/pull/5221)
- [**feature**][**eager**][**op**][**python**][**interface**] Dev modules ctc loss [5168](https://github.com/Oneflow-Inc/oneflow/pull/5168)
- [**bottleneck**][**bug**][**documentation**][**python**][**interface**] Fix meshgrid test bug [5208](https://github.com/Oneflow-Inc/oneflow/pull/5208)
- [**eager**][**documentation**][**python**][**interface**] Rename CosineScheduler to CosineAnnealingLR [5112](https://github.com/Oneflow-Inc/oneflow/pull/5112)
- [**feature**][**eager**][**python**][**interface**] Add meshgrid module [5205](https://github.com/Oneflow-Inc/oneflow/pull/5205)
- [**enhancement**][**feature**][**bug**][**op**][**python**] support bias in conv2d's parameter list [5322](https://github.com/Oneflow-Inc/oneflow/pull/5322)
- [**eager**][**documentation**][**api**][**python**][**interface**] add not_equal, greater_equal and less_equal module [5350](https://github.com/Oneflow-Inc/oneflow/pull/5350)
- [**enhancement**][**eager**][**python**] refine pow module and its test [5319](https://github.com/Oneflow-Inc/oneflow/pull/5319)
- [**enhancement**][**eager**][**op**][**python**] Add triu op [5329](https://github.com/Oneflow-Inc/oneflow/pull/5329)
- [**enhancement**][**bug**][**python**] Fix optimizer for not supporting all kinds of iterables [5355](https://github.com/Oneflow-Inc/oneflow/pull/5355)
- [**bug**][**python**][**interface**] raise IndexError in get_canonical_index to support for loop [5345](https://github.com/Oneflow-Inc/oneflow/pull/5345)
- [**bug**][**python**][**interface**] tensor slice assign supports broadcasting [5344](https://github.com/Oneflow-Inc/oneflow/pull/5344)
- [**enhancement**][**op**][**python**] add cpu group conv logic [5314](https://github.com/Oneflow-Inc/oneflow/pull/5314)
- [**enhancement**][**python**] Add 'nn.Mish' module and corresponding functions [5310](https://github.com/Oneflow-Inc/oneflow/pull/5310)
- [**enhancement**][**build**][**python**] Remove ONNX from setup py [5297](https://github.com/Oneflow-Inc/oneflow/pull/5297)
- [**enhancement**][**python**][**interface**] [add] zeropad2d [5278](https://github.com/Oneflow-Inc/oneflow/pull/5278)
- [**feature**][**system**][**python**][**interface**] Lazy nn.Graph FeedInputOpExpr [5458](https://github.com/Oneflow-Inc/oneflow/pull/5458)
- [**feature**][**python**][**interface**] integrate nn.image.flip [5411](https://github.com/Oneflow-Inc/oneflow/pull/5411)
- [**bug**][**python**] Fix issues in point of MultiClientSession [5469](https://github.com/Oneflow-Inc/oneflow/pull/5469)
- [**enhancement**][**bug**][**python**] update HasAllMultiClientEnvVars() [5459](https://github.com/Oneflow-Inc/oneflow/pull/5459)
- [**enhancement**][**python**] Add in_top_k function [5428](https://github.com/Oneflow-Inc/oneflow/pull/5428)
- [**enhancement**][**python**] Dev add docstring [5449](https://github.com/Oneflow-Inc/oneflow/pull/5449)
- [**feature**][**api**][**python**] MultiClientSession [5407](https://github.com/Oneflow-Inc/oneflow/pull/5407)
- [**documentation**][**python**] remove --user [5431](https://github.com/Oneflow-Inc/oneflow/pull/5431)
- [**feature**][**python**][**interface**] nn.Graph python [5309](https://github.com/Oneflow-Inc/oneflow/pull/5309)
- [**feature**][**python**][**interface**] Fea/nn graph/graph name [5413](https://github.com/Oneflow-Inc/oneflow/pull/5413)
- [**bug**][**python**][**interface**] rm nn.Graph.train [5424](https://github.com/Oneflow-Inc/oneflow/pull/5424)
- [**op**][**documentation**][**api**][**python**][**interface**] add bernoulli module [5353](https://github.com/Oneflow-Inc/oneflow/pull/5353)
- [**enhancement**][**python**] flow.S/B/P [5306](https://github.com/Oneflow-Inc/oneflow/pull/5306)
- [**enhancement**][**documentation**][**python**] Add instruction on upgrade pip [5400](https://github.com/Oneflow-Inc/oneflow/pull/5400)
- [**enhancement**][**python**] Rm oneflow export and experimental [5589](https://github.com/Oneflow-Inc/oneflow/pull/5589)
- [**bug**][**python**] Fix nn.graph.utils module conflict [5598](https://github.com/Oneflow-Inc/oneflow/pull/5598)
- [**feature**][**ci**][**python**] Update autotest framework [5520](https://github.com/Oneflow-Inc/oneflow/pull/5520)
- [**enhancement**][**python**] copy of_proto_python_dir to compatible_single_client_python [5539](https://github.com/Oneflow-Inc/oneflow/pull/5539)
- [**enhancement**][**api**][**python**] del default env init [5537](https://github.com/Oneflow-Inc/oneflow/pull/5537)
- [**enhancement**][**python**] Fix single client using same glog file [5535](https://github.com/Oneflow-Inc/oneflow/pull/5535)
- [**bug**][**api**][**python**] Fix Session TryClose [5531](https://github.com/Oneflow-Inc/oneflow/pull/5531)
- [**enhancement**][**feature**][**python**] split vector-matrix norm [5478](https://github.com/Oneflow-Inc/oneflow/pull/5478)
- [**feature**][**eager**][**op**][**python**][**interface**] Add more upsample kernel [5382](https://github.com/Oneflow-Inc/oneflow/pull/5382)
- [**enhancement**][**feature**][**test**][**python**] add torchstyle unittest [5489](https://github.com/Oneflow-Inc/oneflow/pull/5489)
- [**feature**][**system**][**python**] nn.Graph with training [5662](https://github.com/Oneflow-Inc/oneflow/pull/5662)
- [**enhancement**][**feature**][**python**] Fea/nn graph/block proxy func [5727](https://github.com/Oneflow-Inc/oneflow/pull/5727)
- [**enhancement**][**api**][**python**] consistent_tensor_to_api [5703](https://github.com/Oneflow-Inc/oneflow/pull/5703)
- [**feature**][**eager**][**op**][**python**] Dev Align torch avgpool [5610](https://github.com/Oneflow-Inc/oneflow/pull/5610)
- [**enhancement**][**python**] fix circular deps of sbp python module [5706](https://github.com/Oneflow-Inc/oneflow/pull/5706)
- [**documentation**][**python**] [part5]Remove singleclient outdated api [5674](https://github.com/Oneflow-Inc/oneflow/pull/5674)
- [**enhancement**][**python**] [part4]Remove singleclient outdated api [5672](https://github.com/Oneflow-Inc/oneflow/pull/5672)
- [**bug**][**op**][**python**] remove outdated code in conv3d [5696](https://github.com/Oneflow-Inc/oneflow/pull/5696)
- [**enhancement**][**test**][**python**] enlarge tolerance of dataloader test [5689](https://github.com/Oneflow-Inc/oneflow/pull/5689)
- [**enhancement**][**test**][**python**] add autotest for some math ops [5646](https://github.com/Oneflow-Inc/oneflow/pull/5646)
- [**feature**][**python**] nn.Graph optimizer part 2: add L2, pass job complete, refactor [5604](https://github.com/Oneflow-Inc/oneflow/pull/5604)
- [**enhancement**][**python**] Add clip_grad_norm [5299](https://github.com/Oneflow-Inc/oneflow/pull/5299)
- [**purge**][**python**] Remove Single-Client API in oneflow default python [5827](https://github.com/Oneflow-Inc/oneflow/pull/5827)
- [**bug**][**python**] Fix ddp grad size [5834](https://github.com/Oneflow-Inc/oneflow/pull/5834)
- [**enhancement**][**feature**][**python**] Dev RMSprop graph conf [5768](https://github.com/Oneflow-Inc/oneflow/pull/5768)
- [**enhancement**][**purge**][**eager**][**python**] remove scale arg in optimizer [5821](https://github.com/Oneflow-Inc/oneflow/pull/5821)
- [**enhancement**][**feature**][**python**] graph/block io check [5803](https://github.com/Oneflow-Inc/oneflow/pull/5803)
- [**enhancement**][**feature**][**python**] Dev adam graph conf [5709](https://github.com/Oneflow-Inc/oneflow/pull/5709)
- [**purge**][**python**] [part10]Remove singleclient outdated api [5756](https://github.com/Oneflow-Inc/oneflow/pull/5756)
- [**feature**][**api**][**python**] better repr of nn.Graph for debug [5762](https://github.com/Oneflow-Inc/oneflow/pull/5762)
- [**bug**][**python**] fix weight decay in RMSprop [5755](https://github.com/Oneflow-Inc/oneflow/pull/5755)
- [**purge**][**python**] [part9]Remove singleclient outdated api [5752](https://github.com/Oneflow-Inc/oneflow/pull/5752)
- [**purge**][**python**] [part8]Remove singleclient outdated api [5750](https://github.com/Oneflow-Inc/oneflow/pull/5750)
- [**documentation**][**python**] add first batch of methods in oneflow.nn.functional namespace [5693](https://github.com/Oneflow-Inc/oneflow/pull/5693)
- [**purge**][**python**] [part6]Remove singleclient outdated api [5704](https://github.com/Oneflow-Inc/oneflow/pull/5704)
- [**bug**][**python**] use default_generator.seed() as random_seed in init [5721](https://github.com/Oneflow-Inc/oneflow/pull/5721)
- [**bug**][**system**][**python**] ddp broadcast params and buffers [5913](https://github.com/Oneflow-Inc/oneflow/pull/5913)
- [**enhancement**][**test**][**python**] Add consistent tensor requires grad test [5925](https://github.com/Oneflow-Inc/oneflow/pull/5925)
- [**bug**][**python**] wrap flow.nn.init.\* with flow.no_grad() [5932](https://github.com/Oneflow-Inc/oneflow/pull/5932)
- [**feature**][**api**][**python**][**interface**] add clip_grad to optimizer [5817](https://github.com/Oneflow-Inc/oneflow/pull/5817)
- [**enhancement**][**ci**][**op**][**test**][**python**] add randperm with test and docs [5680](https://github.com/Oneflow-Inc/oneflow/pull/5680)
- [**feature**][**api**][**python**] Fea/nn graph/ lr_schedule(and cosine lr_sch) and opt_group [5846](https://github.com/Oneflow-Inc/oneflow/pull/5846)
- [**bug**][**python**] fix bug of SyncOnMasterFn atexit [5909](https://github.com/Oneflow-Inc/oneflow/pull/5909)
- [**purge**][**python**] Delete single client nn modules [6061](https://github.com/Oneflow-Inc/oneflow/pull/6061)
- [**enhancement**][**python**] Move framework.distribute to env [6022](https://github.com/Oneflow-Inc/oneflow/pull/6022)
- [**bug**][**python**] skip sync when abnormally exiting [6025](https://github.com/Oneflow-Inc/oneflow/pull/6025)
- [**feature**][**python**] Fea/nn graph/warmup amp config [5969](https://github.com/Oneflow-Inc/oneflow/pull/5969)
- [**documentation**][**python**] add optimizer api docs [6131](https://github.com/Oneflow-Inc/oneflow/pull/6131)
- [**documentation**][**python**] add_tensor_api_doc [6127](https://github.com/Oneflow-Inc/oneflow/pull/6127)
- [**bug**][**python**] Fix test_grid_sample.py and test_affine_grid.py threshold [6125](https://github.com/Oneflow-Inc/oneflow/pull/6125)
- [**documentation**][**api**][**python**] add doc of graph [6093](https://github.com/Oneflow-Inc/oneflow/pull/6093)
- [**bug**][**python**] Fix make of_format fail in ubuntu [6120](https://github.com/Oneflow-Inc/oneflow/pull/6120)
- [**feature**][**api**][**python**][**interface**] Fea/graph helpers [6088](https://github.com/Oneflow-Inc/oneflow/pull/6088)
- [**enhancement**][**eager**][**python**][**interface**] Use flow.randint in dataloader [6086](https://github.com/Oneflow-Inc/oneflow/pull/6086)
- [**feature**][**eager**][**api**][**python**][**interface**] Import oneflow as torch [6076](https://github.com/Oneflow-Inc/oneflow/pull/6076)
- [**enhancement**][**test**][**api**][**python**][**refactor**] rename OfrecordReader to OFRcordReader [6090](https://github.com/Oneflow-Inc/oneflow/pull/6090)
- [**purge**][**python**][**need-single-client-tests**] Delete single client nn modules [6082](https://github.com/Oneflow-Inc/oneflow/pull/6082)
- [**enhancement**][**python**] flow.load tolerates FileNotFound fault [6083](https://github.com/Oneflow-Inc/oneflow/pull/6083)
- [**feature**][**python**] Fea/pipeline in graph [6105](https://github.com/Oneflow-Inc/oneflow/pull/6105)
- [**enhancement**][**test**][**python**] graph activation checkpointing [6192](https://github.com/Oneflow-Inc/oneflow/pull/6192)
- [**enhancement**][**feature**][**op**][**python**] rnn test [6165](https://github.com/Oneflow-Inc/oneflow/pull/6165)

New in Ops:

- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part2: Add partial unary and math functional apis [5218](https://github.com/Oneflow-Inc/oneflow/pull/5218)
- [**enhancement**][**bug**][**op**][**interface**] Refine deconv kernel [5229](https://github.com/Oneflow-Inc/oneflow/pull/5229)
- [**enhancement**][**op**][**api**][**interface**] add ReflectionPad2d [5172](https://github.com/Oneflow-Inc/oneflow/pull/5172)
- [**feature**][**eager**][**op**][**api**][**interface**] crossentropyloss and nllloss support ignore_index [5195](https://github.com/Oneflow-Inc/oneflow/pull/5195)
- [**feature**][**eager**][**op**][**api**][**interface**] Yejiaojiao/dev bcewithlogitsloss [5173](https://github.com/Oneflow-Inc/oneflow/pull/5173)
- [**bug**][**ci**][**op**] Dev user op set default is_dynamic [5223](https://github.com/Oneflow-Inc/oneflow/pull/5223)
- [**enhancement**][**op**] add magic method for pow [5199](https://github.com/Oneflow-Inc/oneflow/pull/5199)
- [**enhancement**][**op**][**interface**] add cpu version of upsampling [5194](https://github.com/Oneflow-Inc/oneflow/pull/5194)
- [**enhancement**][**bug**][**op**][**api**][**interface**] add ReplicationPad2d [5148](https://github.com/Oneflow-Inc/oneflow/pull/5148)
- [**feature**][**eager**][**op**][**api**][**interface**] add kldivloss module [5155](https://github.com/Oneflow-Inc/oneflow/pull/5155)
- [**feature**][**eager**][**op**][**documentation**][**build**][**api**][**interface**] Add floor module and the corresponding testcases [4964](https://github.com/Oneflow-Inc/oneflow/pull/4964)
- [**enhancement**][**feature**][**op**] Dev conv1d module [5280](https://github.com/Oneflow-Inc/oneflow/pull/5280)
- [**enhancement**][**op**] Add ctc_greedy_decoder op [5294](https://github.com/Oneflow-Inc/oneflow/pull/5294)
- [**enhancement**][**op**][**system**] Dev remove default grad func [5320](https://github.com/Oneflow-Inc/oneflow/pull/5320)
- [**enhancement**][**op**][**system**] Add pad grad func. [5354](https://github.com/Oneflow-Inc/oneflow/pull/5354)
- [**enhancement**][**op**][**system**] Add gradient funcs. [5348](https://github.com/Oneflow-Inc/oneflow/pull/5348)
- [**feature**][**purge**][**bug**][**eager**][**op**][**interface**] fix upsample nearest bug [5347](https://github.com/Oneflow-Inc/oneflow/pull/5347)
- [**enhancement**][**op**][**system**] [Functional] Part7: Migrate pooling ops [5253](https://github.com/Oneflow-Inc/oneflow/pull/5253)
- [**enhancement**][**op**] nvjpeg hardware acc [5240](https://github.com/Oneflow-Inc/oneflow/pull/5240)
- [**enhancement**][**feature**][**ci**][**eager**][**op**][**api**][**interface**] Add bmm module [5334](https://github.com/Oneflow-Inc/oneflow/pull/5334)
- [**enhancement**][**eager**][**op**] Dev image decode eager [5333](https://github.com/Oneflow-Inc/oneflow/pull/5333)
- [**enhancement**][**op**] Optimize softmax warp impl [4977](https://github.com/Oneflow-Inc/oneflow/pull/4977)
- [**enhancement**][**eager**][**op**] Dev tensor buffer eager [5317](https://github.com/Oneflow-Inc/oneflow/pull/5317)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part6: Migrate conv op [5252](https://github.com/Oneflow-Inc/oneflow/pull/5252)
- [**enhancement**][**eager**][**op**] Dev sort eager [5284](https://github.com/Oneflow-Inc/oneflow/pull/5284)
- [**enhancement**][**bug**][**op**][**api**] fix bceloss bug in default weight and reduction [5303](https://github.com/Oneflow-Inc/oneflow/pull/5303)
- [**bug**][**eager**][**op**] remove redundant assert and check [5264](https://github.com/Oneflow-Inc/oneflow/pull/5264)
- [**enhancement**][**bug**][**ci**][**op**] fix bceloss bug about weight [5269](https://github.com/Oneflow-Inc/oneflow/pull/5269)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part5: Migrate nn ops [5249](https://github.com/Oneflow-Inc/oneflow/pull/5249)
- [**enhancement**][**eager**][**op**] Dev argsort eager [5273](https://github.com/Oneflow-Inc/oneflow/pull/5273)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part4: Migrate array ops [5247](https://github.com/Oneflow-Inc/oneflow/pull/5247)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part3: Migrate binary and activation ops [5246](https://github.com/Oneflow-Inc/oneflow/pull/5246)
- [**bug**][**ci**][**op**][**test**] Dev fix rmsprop ci fail [5481](https://github.com/Oneflow-Inc/oneflow/pull/5481)
- [**enhancement**][**op**] add inplace method: Tensor.sin\_ [5471](https://github.com/Oneflow-Inc/oneflow/pull/5471)
- [**bug**][**op**] hotfix image_batch_align [5461](https://github.com/Oneflow-Inc/oneflow/pull/5461)
- [**enhancement**][**eager**][**op**][**interface**] Dev maxpool series op 123d [5244](https://github.com/Oneflow-Inc/oneflow/pull/5244)
- [**bug**][**op**] fix pool gpu kernel [5446](https://github.com/Oneflow-Inc/oneflow/pull/5446)
- [**feature**][**eager**][**op**][**api**][**interface**] add pixelshufflev2 module [5383](https://github.com/Oneflow-Inc/oneflow/pull/5383)
- [**enhancement**][**feature**][**ci**][**eager**][**op**][**documentation**][**api**][**interface**] Add flow xxx and tensor xxx autotest [5386](https://github.com/Oneflow-Inc/oneflow/pull/5386)
- [**enhancement**][**feature**][**eager**][**op**][**api**][**interface**] Modules chunk [5324](https://github.com/Oneflow-Inc/oneflow/pull/5324)
- [**enhancement**][**eager**][**op**] add image normalize for eager [5402](https://github.com/Oneflow-Inc/oneflow/pull/5402)
- [**enhancement**][**eager**][**op**] Dev batch align module [5401](https://github.com/Oneflow-Inc/oneflow/pull/5401)
- [**enhancement**][**eager**][**op**] add coco reader module [5391](https://github.com/Oneflow-Inc/oneflow/pull/5391)
- [**enhancement**][**wip**][**op**] Restruct Elementwise kernel [4130](https://github.com/Oneflow-Inc/oneflow/pull/4130)
- [**bug**][**op**] Fix DecodeRandom reuse mem [5606](https://github.com/Oneflow-Inc/oneflow/pull/5606)
- [**enhancement**][**op**] Align pytorch maxpool [5525](https://github.com/Oneflow-Inc/oneflow/pull/5525)
- [**enhancement**][**bottleneck**][**eager**][**op**][**api**] implementation of constantpad-3d op [5529](https://github.com/Oneflow-Inc/oneflow/pull/5529)
- [**enhancement**][**eager**][**op**] Add scale size for resize [5509](https://github.com/Oneflow-Inc/oneflow/pull/5509)
- [**enhancement**][**op**][**api**][**refactor**] Dev optimize tensor setitem [5501](https://github.com/Oneflow-Inc/oneflow/pull/5501)
- [**enhancement**][**op**] register uint8 dtypeto support dataloader [5499](https://github.com/Oneflow-Inc/oneflow/pull/5499)
- [**enhancement**][**op**] Add unique.cuh [5487](https://github.com/Oneflow-Inc/oneflow/pull/5487)
- [**enhancement**][**op**][**api**][**interface**] Dev ofrecord auto truncating [5412](https://github.com/Oneflow-Inc/oneflow/pull/5412)
- [**feature**][**op**][**system**][**interface**] Feat: LazyInterpret::ApplyImpl support SourceUserOpExpr and Copy [5711](https://github.com/Oneflow-Inc/oneflow/pull/5711)
- [**enhancement**][**op**][**interface**] Dev logical_and/or modules [5636](https://github.com/Oneflow-Inc/oneflow/pull/5636)
- [**enhancement**][**op**] support any number positional arguments for ones and zeros op [5698](https://github.com/Oneflow-Inc/oneflow/pull/5698)
- [**enhancement**][**feature**][**eager**][**op**] Add conv3d Module [5327](https://github.com/Oneflow-Inc/oneflow/pull/5327)
- [**feature**][**eager**][**op**][**api**][**interface**] add batchnorm3d module [5631](https://github.com/Oneflow-Inc/oneflow/pull/5631)
- [**bug**][**eager**][**op**] fix reduce min max backward bug [5651](https://github.com/Oneflow-Inc/oneflow/pull/5651)
- [**enhancement**][**op**] Debug dim scatter [5371](https://github.com/Oneflow-Inc/oneflow/pull/5371)
- [**enhancement**][**op**][**interface**] Dev eye [5583](https://github.com/Oneflow-Inc/oneflow/pull/5583)
- [**enhancement**][**eager**][**op**] Dev minimum maximum [5576](https://github.com/Oneflow-Inc/oneflow/pull/5576)
- [**enhancement**][**op**] Restruct activation grad op [5669](https://github.com/Oneflow-Inc/oneflow/pull/5669)
- [**enhancement**][**feature**][**eager**][**op**] Rewrite activation function [5465](https://github.com/Oneflow-Inc/oneflow/pull/5465)
- [**bug**][**op**][**documentation**] add oneflow.cat for documentation [5621](https://github.com/Oneflow-Inc/oneflow/pull/5621)
- [**enhancement**][**op**] Lcy logsoftmax [5746](https://github.com/Oneflow-Inc/oneflow/pull/5746)
- [**feature**][**op**][**need-simple-ci**] Feat empty op [5659](https://github.com/Oneflow-Inc/oneflow/pull/5659)
- [**enhancement**][**eager**][**op**] Dev split [5714](https://github.com/Oneflow-Inc/oneflow/pull/5714)
- [**enhancement**][**op**][**interface**] add index_select op [5661](https://github.com/Oneflow-Inc/oneflow/pull/5661)
- [**bug**][**op**] fix nvjpeg hw acc [5851](https://github.com/Oneflow-Inc/oneflow/pull/5851)
- [**enhancement**][**op**] Remove move in conv_cudnn [5828](https://github.com/Oneflow-Inc/oneflow/pull/5828)
- [**enhancement**][**op**][**interface**] Dev logical_xor module [5694](https://github.com/Oneflow-Inc/oneflow/pull/5694)
- [**bug**][**eager**][**op**] fix squeeze [5808](https://github.com/Oneflow-Inc/oneflow/pull/5808)
- [**enhancement**][**op**] Get parallel_id and parallel_num through rank and world size in DDP [5717](https://github.com/Oneflow-Inc/oneflow/pull/5717)
- [**bug**][**eager**][**op**] delete interpolate int type [5805](https://github.com/Oneflow-Inc/oneflow/pull/5805)
- [**bug**][**op**] Fix bug in scatter [5743](https://github.com/Oneflow-Inc/oneflow/pull/5743)
- [**enhancement**][**op**] Refactor: remove module not required, call function directly [5754](https://github.com/Oneflow-Inc/oneflow/pull/5754)
- [**enhancement**][**op**] Remove modules not required(tan, erfc, log1p, scatter_nd) [5791](https://github.com/Oneflow-Inc/oneflow/pull/5791)
- [**enhancement**][**op**] Refactor scatter, clamp and pow in cpp instead of in python [5715](https://github.com/Oneflow-Inc/oneflow/pull/5715)
- [**enhancement**][**op**] Rm useless code in gather files [5687](https://github.com/Oneflow-Inc/oneflow/pull/5687)
- [**enhancement**][**eager**][**op**] change flip_code to scalar [5786](https://github.com/Oneflow-Inc/oneflow/pull/5786)
- [**enhancement**][**bug**][**op**][**interface**] fix upsample bug [5753](https://github.com/Oneflow-Inc/oneflow/pull/5753)
- [**bug**][**op**][**interface**] Quick fix Lazy nn.Graph input/output OpConf.BlobConf.is_dynamic [5767](https://github.com/Oneflow-Inc/oneflow/pull/5767)
- [**enhancement**][**bug**][**eager**][**op**] fix argwhere 0-dim bug [5760](https://github.com/Oneflow-Inc/oneflow/pull/5760)
- [**enhancement**][**eager**][**op**] delete unused code [5744](https://github.com/Oneflow-Inc/oneflow/pull/5744)
- [**feature**][**op**] Export fused_scale_tril op [5933](https://github.com/Oneflow-Inc/oneflow/pull/5933)
- [**bug**][**op**] Fix backward bug in 3d [5908](https://github.com/Oneflow-Inc/oneflow/pull/5908)
- [**bug**][**op**] Fix one_hot api limit [5927](https://github.com/Oneflow-Inc/oneflow/pull/5927)
- [**enhancement**][**eager**][**op**] Dev where scalar [5797](https://github.com/Oneflow-Inc/oneflow/pull/5797)
- [**bug**][**op**] fix grad error [5914](https://github.com/Oneflow-Inc/oneflow/pull/5914)
- [**feature**][**bug**][**op**] Fix inplace op circle reference bug [5910](https://github.com/Oneflow-Inc/oneflow/pull/5910)
- [**enhancement**][**op**] Move the judgment content to c++， And add scalar fmod [5854](https://github.com/Oneflow-Inc/oneflow/pull/5854)
- [**enhancement**][**op**] Support combined_margin_loss op in flow.nn.modules [5830](https://github.com/Oneflow-Inc/oneflow/pull/5830)
- [**enhancement**][**op**][**api**][**interface**] functional_one_hot [5315](https://github.com/Oneflow-Inc/oneflow/pull/5315)
- [**enhancement**][**op**] Dev scalar op [5778](https://github.com/Oneflow-Inc/oneflow/pull/5778)
- [**bug**][**eager**][**op**] fix gather kernel 0 shape [5888](https://github.com/Oneflow-Inc/oneflow/pull/5888)
- [**enhancement**][**op**] add l2_normalize for mutl-client interfaces [5859](https://github.com/Oneflow-Inc/oneflow/pull/5859)
- [**feature**][**op**] Export function softmax_cross_entropy [6056](https://github.com/Oneflow-Inc/oneflow/pull/6056)
- [**enhancement**][**op**] Add int attr for functional adaptive average pool [6059](https://github.com/Oneflow-Inc/oneflow/pull/6059)
- [**enhancement**][**op**][**interface**] dev full op [5955](https://github.com/Oneflow-Inc/oneflow/pull/5955)
- [**bug**][**eager**][**op**] fix 0dim inplace add [6029](https://github.com/Oneflow-Inc/oneflow/pull/6029)
- [**feature**][**op**][**system**][**interface**] Feat: nn.Graph image gpu decoder [6014](https://github.com/Oneflow-Inc/oneflow/pull/6014)
- [**enhancement**][**op**][**interface**] dev optim_optim_lr_scheduler_multisteplr [5975](https://github.com/Oneflow-Inc/oneflow/pull/5975)
- [**enhancement**][**op**] NopKernel [6035](https://github.com/Oneflow-Inc/oneflow/pull/6035)
- [**enhancement**][**eager**][**op**][**api**] Dev tril op [6005](https://github.com/Oneflow-Inc/oneflow/pull/6005)
- [**enhancement**][**op**] dev unfold and fold [5675](https://github.com/Oneflow-Inc/oneflow/pull/5675)
- [**enhancement**][**op**] ResNet CUDA Graphs [6018](https://github.com/Oneflow-Inc/oneflow/pull/6018)
- [**enhancement**][**feature**][**op**] add broadcast pow [6013](https://github.com/Oneflow-Inc/oneflow/pull/6013)
- [**enhancement**][**op**][**interface**] init of op diag [5298](https://github.com/Oneflow-Inc/oneflow/pull/5298)
- [**op**][**documentation**][**api**] Fix api document bug [6009](https://github.com/Oneflow-Inc/oneflow/pull/6009)
- [**enhancement**][**op**] Dev fused functional [5954](https://github.com/Oneflow-Inc/oneflow/pull/5954)
- [**bug**][**op**][**build**] Add nvcc flag -Werror cross-execution-space-call [6002](https://github.com/Oneflow-Inc/oneflow/pull/6002)
- [**bug**][**op**] Fix Normalization grad function [5993](https://github.com/Oneflow-Inc/oneflow/pull/5993)
- [**enhancement**][**feature**][**eager**][**op**][**test**][**interface**] Add fused self attention [5966](https://github.com/Oneflow-Inc/oneflow/pull/5966)
- [**enhancement**][**bug**][**ci**][**eager**][**op**][**api**][**interface**] Try to fix var bug [5973](https://github.com/Oneflow-Inc/oneflow/pull/5973)
- [**enhancement**][**feature**][**eager**][**op**][**interface**] add prod op [5867](https://github.com/Oneflow-Inc/oneflow/pull/5867)
- [**enhancement**][**eager**][**op**][**api**] add glu op [6065](https://github.com/Oneflow-Inc/oneflow/pull/6065)
- [**enhancement**][**op**] Align Torch.nn.functional poolXd [6184](https://github.com/Oneflow-Inc/oneflow/pull/6184)
- [**bug**][**eager**][**op**] fix backward index for gamma beta [6149](https://github.com/Oneflow-Inc/oneflow/pull/6149)
- [**bug**][**op**][**system**] Fix BroadcastMatmulGrad bug [6168](https://github.com/Oneflow-Inc/oneflow/pull/6168)
- [**enhancement**][**op**][**api**] Add Int support for functional.avg/maxpool [6174](https://github.com/Oneflow-Inc/oneflow/pull/6174)
- [**bug**][**eager**][**op**][**api**][**interface**] align dropout api name with pytorch [6170](https://github.com/Oneflow-Inc/oneflow/pull/6170)
- [**enhancement**][**op**] support inplace operation for hardsigmoid [6137](https://github.com/Oneflow-Inc/oneflow/pull/6137)
- [**enhancement**][**bug**][**op**] Fix do bias correction in Adam/AdamW [5960](https://github.com/Oneflow-Inc/oneflow/pull/5960)
- [**bug**][**eager**][**op**][**api**][**interface**] fix repeat 0-dim tensor bug [6150](https://github.com/Oneflow-Inc/oneflow/pull/6150)
- [**enhancement**][**bug**][**op**] Fix select_first_grad bug [6142](https://github.com/Oneflow-Inc/oneflow/pull/6142)
- [**bug**][**ci**][**eager**][**op**][**documentation**][**interface**] Add clipgrad doc and contiguous [6130](https://github.com/Oneflow-Inc/oneflow/pull/6130)
- [**bug**][**op**] Fix eager optim dynamic attr bug [6111](https://github.com/Oneflow-Inc/oneflow/pull/6111)
- [**enhancement**][**op**] Support grid_sample and affine_grid operator [6038](https://github.com/Oneflow-Inc/oneflow/pull/6038)
- [**op**][**documentation**] Export apis for documentation [6068](https://github.com/Oneflow-Inc/oneflow/pull/6068)
- [**enhancement**][**feature**][**bug**][**ci**][**eager**][**op**][**documentation**][**interface**] transfer python function to c++ method [6114](https://github.com/Oneflow-Inc/oneflow/pull/6114)
- [**op**][**documentation**] Dev functional batch_gather [6233](https://github.com/Oneflow-Inc/oneflow/pull/6233)
- [**enhancement**][**op**][**test**] fix cross_entropy_loss and its test [5799](https://github.com/Oneflow-Inc/oneflow/pull/5799)
- [**bug**][**op**] Use attr nd_sbp to check consistent [6222](https://github.com/Oneflow-Inc/oneflow/pull/6222)
- [**enhancement**][**op**] Dev fused bn functional [6077](https://github.com/Oneflow-Inc/oneflow/pull/6077)
- [**enhancement**][**op**] support default value in intlist [6201](https://github.com/Oneflow-Inc/oneflow/pull/6201)
- [**bug**][**op**] fix sparse_softmax get_nd_sbp [6203](https://github.com/Oneflow-Inc/oneflow/pull/6203)
- [**bug**][**op**] Fix bug in model fused update [6197](https://github.com/Oneflow-Inc/oneflow/pull/6197)
- [**enhancement**][**op**][**system**][**refactor**] Optimize tensor getitem. [5433](https://github.com/Oneflow-Inc/oneflow/pull/5433)

New in Eager:

- [**enhancement**][**eager**][**interface**] Reconstruct module files [5251](https://github.com/Oneflow-Inc/oneflow/pull/5251)
- [**bug**][**eager**][**documentation**][**interface**] Fix conv module bug [5245](https://github.com/Oneflow-Inc/oneflow/pull/5245)
- [**bug**][**ci**][**eager**][**interface**] Fix bce withlogitloss ci error [5237](https://github.com/Oneflow-Inc/oneflow/pull/5237)
- [**feature**][**eager**][**api**][**interface**] module BCELoss [5144](https://github.com/Oneflow-Inc/oneflow/pull/5144)
- [**enhancement**][**feature**][**eager**][**api**][**interface**] Dev norm op [5178](https://github.com/Oneflow-Inc/oneflow/pull/5178)
- [**enhancement**][**bug**][**eager**] Fix stack module [5222](https://github.com/Oneflow-Inc/oneflow/pull/5222)
- [**enhancement**][**feature**][**eager**] Support different dtype of equal module [5214](https://github.com/Oneflow-Inc/oneflow/pull/5214)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**][**interface**] Add nllloss backward [5210](https://github.com/Oneflow-Inc/oneflow/pull/5210)
- [**enhancement**][**eager**][**api**][**upload-core**] Decouple FileSystem and IOConf [5162](https://github.com/Oneflow-Inc/oneflow/pull/5162)
- [**enhancement**][**ci**][**eager**] Set lower precision avoid ci failing [5200](https://github.com/Oneflow-Inc/oneflow/pull/5200)
- [**eager**][**documentation**] Add hint when apply FunctionNode second time [5369](https://github.com/Oneflow-Inc/oneflow/pull/5369)
- [**enhancement**][**feature**][**bug**][**ci**][**eager**][**documentation**][**api**] Fix upsample bilinear bug [5366](https://github.com/Oneflow-Inc/oneflow/pull/5366)
- [**bug**][**eager**] Fix not contiguous ndarray to tensor bug [5351](https://github.com/Oneflow-Inc/oneflow/pull/5351)
- [**enhancement**][**eager**][**system**] Infer consistent tensor meta [5118](https://github.com/Oneflow-Inc/oneflow/pull/5118)
- [**feature**][**eager**] Feat graph autograd engine [5296](https://github.com/Oneflow-Inc/oneflow/pull/5296)
- [**enhancement**][**eager**][**interface**] Dev type as module [5349](https://github.com/Oneflow-Inc/oneflow/pull/5349)
- [**feature**][**eager**][**documentation**][**api**][**interface**] Add new ones module [5342](https://github.com/Oneflow-Inc/oneflow/pull/5342)
- [**enhancement**][**bug**][**eager**] Fix logical slice assign dtype [5339](https://github.com/Oneflow-Inc/oneflow/pull/5339)
- [**bug**][**ci**][**eager**][**documentation**][**api**][**interface**] Fix where module bug [5300](https://github.com/Oneflow-Inc/oneflow/pull/5300)
- [**bug**][**ci**][**eager**][**documentation**][**api**] Fix l1loss ci error [5307](https://github.com/Oneflow-Inc/oneflow/pull/5307)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**][**interface**] Qi's First Edit of deleting "print" and ".numpy" [5129](https://github.com/Oneflow-Inc/oneflow/pull/5129)
- [**feature**][**eager**][**refactor**] Separate autograd meta to tensor [5267](https://github.com/Oneflow-Inc/oneflow/pull/5267)
- [**feature**][**eager**][**api**][**interface**] add tile module [5234](https://github.com/Oneflow-Inc/oneflow/pull/5234)
- [**enhancement**][**eager**] Release lambda function to reuse tensor memory [5266](https://github.com/Oneflow-Inc/oneflow/pull/5266)
- [**feature**][**bug**][**eager**][**documentation**] Fix default value not set bug [5483](https://github.com/Oneflow-Inc/oneflow/pull/5483)
- [**enhancement**][**eager**][**interface**] [Add] gather_nd scatter_nd [5422](https://github.com/Oneflow-Inc/oneflow/pull/5422)
- [**enhancement**][**bug**][**eager**] fix param [5473](https://github.com/Oneflow-Inc/oneflow/pull/5473)
- [**bug**][**eager**] Fix Tensor.grad setter bug [5462](https://github.com/Oneflow-Inc/oneflow/pull/5462)
- [**enhancement**][**eager**] Rename now_grad_arg to current_grad [5466](https://github.com/Oneflow-Inc/oneflow/pull/5466)
- [**eager**][**test**][**documentation**][**interface**] Add autotest part1 [5436](https://github.com/Oneflow-Inc/oneflow/pull/5436)
- [**enhancement**][**eager**] Use functional copy instead of op_builder [5460](https://github.com/Oneflow-Inc/oneflow/pull/5460)
- [**bottleneck**][**bug**][**eager**][**interface**] fix -1 index not support bug [5448](https://github.com/Oneflow-Inc/oneflow/pull/5448)
- [**bug**][**ci**][**eager**][**documentation**][**api**] Fix concat backward bug [5443](https://github.com/Oneflow-Inc/oneflow/pull/5443)
- [**enhancement**][**bug**][**ci**][**eager**] Add autograd engine warning [5444](https://github.com/Oneflow-Inc/oneflow/pull/5444)
- [**feature**][**eager**][**api**][**interface**] Smoothl1loss [5256](https://github.com/Oneflow-Inc/oneflow/pull/5256)
- [**enhancement**][**bottleneck**][**eager**] remove device dtype params [5434](https://github.com/Oneflow-Inc/oneflow/pull/5434)
- [**bug**][**ci**][**eager**][**documentation**][**interface**] Delete maxpool failed test [5409](https://github.com/Oneflow-Inc/oneflow/pull/5409)
- [**enhancement**][**eager**][**api**] Add tensor grad assginment [5379](https://github.com/Oneflow-Inc/oneflow/pull/5379)
- [**enhancement**][**bug**][**eager**] fix-abs [5398](https://github.com/Oneflow-Inc/oneflow/pull/5398)
- [**enhancement**][**bug**][**eager**][**interface**] Fix bn track running stats [5393](https://github.com/Oneflow-Inc/oneflow/pull/5393)
- [**enhancement**][**bug**][**eager**] Support uint dtype of constant op [5396](https://github.com/Oneflow-Inc/oneflow/pull/5396)
- [**enhancement**][**bug**][**eager**][**documentation**][**interface**] Delete useless code upsample [5392](https://github.com/Oneflow-Inc/oneflow/pull/5392)
- [**enhancement**][**ci**][**eager**][**interface**] add flow.view [5301](https://github.com/Oneflow-Inc/oneflow/pull/5301)
- [**enhancement**][**bug**][**ci**][**eager**][**api**][**interface**] Add masked select module [5356](https://github.com/Oneflow-Inc/oneflow/pull/5356)
- [**bug**][**eager**][**interface**] Fix batchnorm backward bug [5602](https://github.com/Oneflow-Inc/oneflow/pull/5602)
- [**enhancement**][**eager**] Support weight_dacay(l2 actually) [5587](https://github.com/Oneflow-Inc/oneflow/pull/5587)
- [**feature**][**eager**][**documentation**][**api**] Add new autotest [5588](https://github.com/Oneflow-Inc/oneflow/pull/5588)
- [**enhancement**][**eager**][**documentation**][**api**] Dev fmod [5404](https://github.com/Oneflow-Inc/oneflow/pull/5404)
- [**feature**][**eager**] Support inplace add [5432](https://github.com/Oneflow-Inc/oneflow/pull/5432)
- [**feature**][**eager**][**interface**] Feat tensor stride property [5543](https://github.com/Oneflow-Inc/oneflow/pull/5543)
- [**enhancement**][**feature**][**eager**][**documentation**][**api**] Add flip module [5541](https://github.com/Oneflow-Inc/oneflow/pull/5541)
- [**feature**][**eager**] Feat module repr [5486](https://github.com/Oneflow-Inc/oneflow/pull/5486)
- [**enhancement**][**bottleneck**][**bug**][**eager**][**interface**] Fix maxpool1d params [5493](https://github.com/Oneflow-Inc/oneflow/pull/5493)
- [**enhancement**][**feature**][**eager**][**interface**] Dev flow.utils.data part1 [5406](https://github.com/Oneflow-Inc/oneflow/pull/5406)
- [**bug**][**eager**][**api**] Fix tensor getitem bug [5474](https://github.com/Oneflow-Inc/oneflow/pull/5474)
- [**enhancement**][**eager**][**need-simple-ci**] export datasets interface [5691](https://github.com/Oneflow-Inc/oneflow/pull/5691)
- [**enhancement**][**eager**][**system**] rebase [5601](https://github.com/Oneflow-Inc/oneflow/pull/5601)
- [**enhancement**][**eager**][**test**] added nn.RecordBytesDecoder with its test [5475](https://github.com/Oneflow-Inc/oneflow/pull/5475)
- [**enhancement**][**feature**][**eager**][**need-simple-ci**] 0-dim tensor support [5552](https://github.com/Oneflow-Inc/oneflow/pull/5552)
- [**enhancement**][**bug**][**eager**] rewrite slice_update backward [5677](https://github.com/Oneflow-Inc/oneflow/pull/5677)
- [**enhancement**][**bug**][**eager**][**interface**] align view input style with torch [5676](https://github.com/Oneflow-Inc/oneflow/pull/5676)
- [**enhancement**][**eager**][**interface**][**need-simple-ci**] add autotests for modules [5666](https://github.com/Oneflow-Inc/oneflow/pull/5666)
- [**enhancement**][**bottleneck**][**eager**][**interface**] Dev constantpad1d op [5579](https://github.com/Oneflow-Inc/oneflow/pull/5579)
- [**enhancement**][**eager**][**api**][**interface**] Restruct MathOps AutoTest [5654](https://github.com/Oneflow-Inc/oneflow/pull/5654)
- [**enhancement**][**bug**][**ci**][**eager**] Fix flip bug [5657](https://github.com/Oneflow-Inc/oneflow/pull/5657)
- [**bug**][**eager**][**api**][**interface**] Fix expand module bug [5650](https://github.com/Oneflow-Inc/oneflow/pull/5650)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**] Fix repeat bug [5633](https://github.com/Oneflow-Inc/oneflow/pull/5633)
- [**enhancement**][**eager**][**test**][**api**][**interface**] Add new autotest [5617](https://github.com/Oneflow-Inc/oneflow/pull/5617)
- [**enhancement**][**eager**][**api**][**interface**] Dev flow.utils.data part2 [5500](https://github.com/Oneflow-Inc/oneflow/pull/5500)
- [**enhancement**][**bug**][**eager**] make setitem device match [5835](https://github.com/Oneflow-Inc/oneflow/pull/5835)
- [**bug**][**eager**][**api**][**interface**] align reshape input param with pytorch [5804](https://github.com/Oneflow-Inc/oneflow/pull/5804)
- [**feature**][**bug**][**eager**][**api**] Align where op with torch [5850](https://github.com/Oneflow-Inc/oneflow/pull/5850)
- [**enhancement**][**bug**][**eager**][**api**] Restruct prelu op [5829](https://github.com/Oneflow-Inc/oneflow/pull/5829)
- [**bug**][**eager**][**need-simple-ci**] fix pooling ceil_mode bug [5818](https://github.com/Oneflow-Inc/oneflow/pull/5818)
- [**enhancement**][**eager**] stateful local kernel supports consistent [5789](https://github.com/Oneflow-Inc/oneflow/pull/5789)
- [**bug**][**eager**][**api**][**interface**] Fix argwhere bug [5816](https://github.com/Oneflow-Inc/oneflow/pull/5816)
- [**enhancement**][**eager**][**documentation**][**api**] dev-nonzero [5809](https://github.com/Oneflow-Inc/oneflow/pull/5809)
- [**enhancement**][**feature**][**eager**][**api**] Add fake quantize op [5690](https://github.com/Oneflow-Inc/oneflow/pull/5690)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**] Add api [5663](https://github.com/Oneflow-Inc/oneflow/pull/5663)
- [**enhancement**][**eager**] Refactor consistent infer result [5790](https://github.com/Oneflow-Inc/oneflow/pull/5790)
- [**bug**][**eager**][**need-simple-ci**] skip dataloader test [5780](https://github.com/Oneflow-Inc/oneflow/pull/5780)
- [**bug**][**eager**][**need-simple-ci**] fix 0-dim tensor.fill\_ [5771](https://github.com/Oneflow-Inc/oneflow/pull/5771)
- [**enhancement**][**eager**] Cpu mpi broadcast [5726](https://github.com/Oneflow-Inc/oneflow/pull/5726)
- [**feature**][**eager**] Feat grad mode classes [5956](https://github.com/Oneflow-Inc/oneflow/pull/5956)
- [**enhancement**][**bug**][**eager**] fix wrong names [5951](https://github.com/Oneflow-Inc/oneflow/pull/5951)
- [**enhancement**][**eager**][**system**] Local dep object pool [5953](https://github.com/Oneflow-Inc/oneflow/pull/5953)
- [**enhancement**][**eager**][**interface**] rename OpExprInterpState to AutoGradCaptureState [5918](https://github.com/Oneflow-Inc/oneflow/pull/5918)
- [**bug**][**eager**] Fix linear bug [5945](https://github.com/Oneflow-Inc/oneflow/pull/5945)
- [**bug**][**eager**] Fix tensor_meta update bug [5924](https://github.com/Oneflow-Inc/oneflow/pull/5924)
- [**enhancement**][**eager**] use flow.randperm [5928](https://github.com/Oneflow-Inc/oneflow/pull/5928)
- [**enhancement**][**eager**] consistent init/save/load [5896](https://github.com/Oneflow-Inc/oneflow/pull/5896)
- [**enhancement**][**bug**][**eager**][**documentation**][**interface**] Restruct sort and argsort op [5911](https://github.com/Oneflow-Inc/oneflow/pull/5911)
- [**enhancement**][**bug**][**eager**][**interface**] Try to fix the problem that the insightface cannot converge。 [5906](https://github.com/Oneflow-Inc/oneflow/pull/5906)
- [**enhancement**][**bug**][**eager**][**interface**] Add autotest [5899](https://github.com/Oneflow-Inc/oneflow/pull/5899)
- [**enhancement**][**eager**] The scheduler thread joins worker threads [5893](https://github.com/Oneflow-Inc/oneflow/pull/5893)
- [**enhancement**][**eager**] Bugfix async callback [5881](https://github.com/Oneflow-Inc/oneflow/pull/5881)
- [**feature**][**eager**] Feat tensor to bool [5836](https://github.com/Oneflow-Inc/oneflow/pull/5836)
- [**bug**][**eager**] Remove inplace broadcast_add [5551](https://github.com/Oneflow-Inc/oneflow/pull/5551)
- [**enhancement**][**eager**] Broadcast consistent shape and dtype [5784](https://github.com/Oneflow-Inc/oneflow/pull/5784)
- [**enhancement**][**eager**] Fix optimizer list parameters input bug [5848](https://github.com/Oneflow-Inc/oneflow/pull/5848)
- [**enhancement**][**eager**][**interface**] Dev flow.utils.data part3 [5644](https://github.com/Oneflow-Inc/oneflow/pull/5644)
- [**enhancement**][**eager**][**api**] Normalize naming of modules [6066](https://github.com/Oneflow-Inc/oneflow/pull/6066)
- [**enhancement**][**feature**][**eager**][**api**][**interface**] add trunc*normal* [6051](https://github.com/Oneflow-Inc/oneflow/pull/6051)
- [**enhancement**][**bug**][**eager**] AutoMatedTest support test module.parameter.grad [6043](https://github.com/Oneflow-Inc/oneflow/pull/6043)
- [**enhancement**][**feature**][**bug**][**eager**] add module **call** kwags [6069](https://github.com/Oneflow-Inc/oneflow/pull/6069)
- [**enhancement**][**eager**][**api**][**interface**] add tensor.item tensor.tolist [6021](https://github.com/Oneflow-Inc/oneflow/pull/6021)
- [**enhancement**][**eager**][**api**][**interface**] Export pool ops api [6047](https://github.com/Oneflow-Inc/oneflow/pull/6047)
- [**enhancement**][**bug**][**eager**][**test**][**documentation**][**interface**] Add more autotest sample [6039](https://github.com/Oneflow-Inc/oneflow/pull/6039)
- [**enhancement**][**bug**][**eager**][**system**] disable cuda_h2d stream [6020](https://github.com/Oneflow-Inc/oneflow/pull/6020)
- [**feature**][**eager**][**test**][**api**][**interface**] Add autotest codegen [6019](https://github.com/Oneflow-Inc/oneflow/pull/6019)
- [**feature**][**eager**][**documentation**] Refactor cosine lr scheduler [6000](https://github.com/Oneflow-Inc/oneflow/pull/6000)
- [**enhancement**][**eager**][**interface**] tensor.cpu/tensor.cuda [5894](https://github.com/Oneflow-Inc/oneflow/pull/5894)
- [**enhancement**][**eager**][**api**] Support consistent_tensor.to(dtype) [5991](https://github.com/Oneflow-Inc/oneflow/pull/5991)
- [**bug**][**eager**][**interface**] remove redundant codes in ModuleDict [5961](https://github.com/Oneflow-Inc/oneflow/pull/5961)
- [**bug**][**eager**] Fix LayerNorm check bug [6196](https://github.com/Oneflow-Inc/oneflow/pull/6196)
- [**enhancement**][**eager**][**api**] Change dropout api [6182](https://github.com/Oneflow-Inc/oneflow/pull/6182)
- [**enhancement**][**good for pr**][**eager**][**api**][**interface**] add: test convert dependency [6023](https://github.com/Oneflow-Inc/oneflow/pull/6023)
- [**enhancement**][**bug**][**eager**][**interface**] Fix autotest codegen bug [6171](https://github.com/Oneflow-Inc/oneflow/pull/6171)
- [**bug**][**eager**] restore instr_local_dep_object_pool_size for nccl [6160](https://github.com/Oneflow-Inc/oneflow/pull/6160)
- [**enhancement**][**eager**][**api**][**interface**] Aligin pooling op functional api names with torch [6163](https://github.com/Oneflow-Inc/oneflow/pull/6163)
- [**feature**][**bug**][**eager**][**api**][**interface**] delete file [6162](https://github.com/Oneflow-Inc/oneflow/pull/6162)
- [**bug**][**eager**] Fix optim load_state_dict bug [6152](https://github.com/Oneflow-Inc/oneflow/pull/6152)
- [**enhancement**][**eager**][**api**] add is_training to dropout functor [6148](https://github.com/Oneflow-Inc/oneflow/pull/6148)
- [**enhancement**][**eager**] Decompose nd sbp boxing [5800](https://github.com/Oneflow-Inc/oneflow/pull/5800)
- [**enhancement**][**eager**] support consistent_tensor.to(copy=True) [6122](https://github.com/Oneflow-Inc/oneflow/pull/6122)
- [**feature**][**eager**] Static grad scaler [6135](https://github.com/Oneflow-Inc/oneflow/pull/6135)
- [**bug**][**eager**] Fix LayerNorm expr bug [6121](https://github.com/Oneflow-Inc/oneflow/pull/6121)
- [**bug**][**eager**][**api**] move numpy c api init in numpy.cpp, make np array contiguous before copying [6117](https://github.com/Oneflow-Inc/oneflow/pull/6117)
- [**enhancement**][**eager**][**refactor**] Remove params from ParamGroup getitem [6096](https://github.com/Oneflow-Inc/oneflow/pull/6096)
- [**enhancement**][**feature**][**eager**] Support tensor and optimizer serialization [6087](https://github.com/Oneflow-Inc/oneflow/pull/6087)
- [**enhancement**][**bug**][**eager**] fix bug about tensor str in nonsymmetric cast and getitem in consist… [6239](https://github.com/Oneflow-Inc/oneflow/pull/6239)
- [**enhancement**][**eager**] Cpu all reduce [5849](https://github.com/Oneflow-Inc/oneflow/pull/5849)
- [**feature**][**eager**] Support assign copy interface [6228](https://github.com/Oneflow-Inc/oneflow/pull/6228)
- [**enhancement**][**eager**][**api**][**interface**] Dev reconstruct pad ops [6223](https://github.com/Oneflow-Inc/oneflow/pull/6223)
- [**enhancement**][**eager**][**api**][**interface**] support flow.cuda.is_available [6124](https://github.com/Oneflow-Inc/oneflow/pull/6124)
- [**bug**][**eager**] make flow.\_C.local_all_reduce sync lanuched [6175](https://github.com/Oneflow-Inc/oneflow/pull/6175)
- [**enhancement**][**eager**] Rename flow to oneflow in user hint [6190](https://github.com/Oneflow-Inc/oneflow/pull/6190)
- [**bug**][**eager**][**tooling**][**test**][**api**][**interface**] Autotest generate input tensor [6206](https://github.com/Oneflow-Inc/oneflow/pull/6206)
- [**enhancement**][**eager**] consistent tensor zeros\_() [6202](https://github.com/Oneflow-Inc/oneflow/pull/6202)
- [**enhancement**][**eager**] Cpu mpi [5865](https://github.com/Oneflow-Inc/oneflow/pull/5865)

Build enhancements:

- [**bug**][**build**] Fix GRPC compilation failure on CMake 3.20 [5255](https://github.com/Oneflow-Inc/oneflow/pull/5255)
- [**bug**][**build**] Refine header file copy [5254](https://github.com/Oneflow-Inc/oneflow/pull/5254)
- [**bug**][**build**] Fix older version CMake doesn't support multiple targets in CLI [5248](https://github.com/Oneflow-Inc/oneflow/pull/5248)
- [**bug**][**build**] Turn off NCCL_STATIC/CUDNN_STATIC when CUDA_STATIC is OFF [5243](https://github.com/Oneflow-Inc/oneflow/pull/5243)
- [**feature**][**build**] Fix support for Ninja and add Ninja build in Simple CI [5236](https://github.com/Oneflow-Inc/oneflow/pull/5236)
- [**enhancement**][**build**] Add cmake option CUDA_STATIC [5164](https://github.com/Oneflow-Inc/oneflow/pull/5164)
- [**bug**][**build**] Fix protobuf debug postfix [5233](https://github.com/Oneflow-Inc/oneflow/pull/5233)
- [**enhancement**][**ci**][**build**] Move default third party dir into build dir [5230](https://github.com/Oneflow-Inc/oneflow/pull/5230)
- [**enhancement**][**build**] Refine protobuf cmake [5216](https://github.com/Oneflow-Inc/oneflow/pull/5216)
- [**enhancement**][**ci**][**build**] Remove transport test main [5215](https://github.com/Oneflow-Inc/oneflow/pull/5215)
- [**enhancement**][**ci**][**build**] Speedup opencv build [5213](https://github.com/Oneflow-Inc/oneflow/pull/5213)
- [**enhancement**][**build**] Support clang [5015](https://github.com/Oneflow-Inc/oneflow/pull/5015)
- [**enhancement**][**documentation**][**build**] Add prefix when creating git archive [5201](https://github.com/Oneflow-Inc/oneflow/pull/5201)
- [**enhancement**][**build**] Add cmake option NCCL_STATIC [5160](https://github.com/Oneflow-Inc/oneflow/pull/5160)
- [**enhancement**][**build**] Refine CMake CUDA version handling [5192](https://github.com/Oneflow-Inc/oneflow/pull/5192)
- [**enhancement**][**build**] Use clang plugin to check Maybe variables are used [5358](https://github.com/Oneflow-Inc/oneflow/pull/5358)
- [**enhancement**][**build**] Add BUILD_BYPRODUCTS for ExternalProject_Add [5316](https://github.com/Oneflow-Inc/oneflow/pull/5316)
- [**enhancement**][**build**] Add cmake init cache to simplify user onboarding [5311](https://github.com/Oneflow-Inc/oneflow/pull/5311)
- [**feature**][**bug**][**build**] Fix macOS support and run macOS build in Simple CI [4947](https://github.com/Oneflow-Inc/oneflow/pull/4947)
- [**enhancement**][**build**] flatbuffers use mirror [5295](https://github.com/Oneflow-Inc/oneflow/pull/5295)
- [**enhancement**][**build**] Don't build test by default [5302](https://github.com/Oneflow-Inc/oneflow/pull/5302)
- [**enhancement**][**build**] Prevent building from scratch when toggle flag BUILD_GIT_VERSION [5259](https://github.com/Oneflow-Inc/oneflow/pull/5259)
- [**enhancement**][**build**] Refine gRPC, glog, gflags cmake for conda [5276](https://github.com/Oneflow-Inc/oneflow/pull/5276)
- [**feature**][**build**] Support XLA with CPU-only [5260](https://github.com/Oneflow-Inc/oneflow/pull/5260)
- [**enhancement**][**ci**][**onnx**][**build**] Remove ONNX from CI [5257](https://github.com/Oneflow-Inc/oneflow/pull/5257)
- [**enhancement**][**build**] Refactor build_wheel to support oneflowinc images [5427](https://github.com/Oneflow-Inc/oneflow/pull/5427)
- [**enhancement**][**build**] Add arg skip_audit in build wheel [5423](https://github.com/Oneflow-Inc/oneflow/pull/5423)
- [**bug**][**build**] hwloc disable shared [5388](https://github.com/Oneflow-Inc/oneflow/pull/5388)
- [**documentation**][**build**] Update readme for autoconf and libtool [5376](https://github.com/Oneflow-Inc/oneflow/pull/5376)
- [**enhancement**][**build**] remove dir python and compatible_single_client_python [5609](https://github.com/Oneflow-Inc/oneflow/pull/5609)
- [**bug**][**build**][**system**] Fix pyyaml version [5594](https://github.com/Oneflow-Inc/oneflow/pull/5594)
- [**enhancement**][**ci**][**build**] force release flags [5574](https://github.com/Oneflow-Inc/oneflow/pull/5574)
- [**bug**][**build**] prevent endless loop [5534](https://github.com/Oneflow-Inc/oneflow/pull/5534)
- [**enhancement**][**build**] Support sccache [5528](https://github.com/Oneflow-Inc/oneflow/pull/5528)
- [**enhancement**][**build**] Add definition for CMAKE_BUILD_TYPE and print cmake_build_type in oneflow doctor [5505](https://github.com/Oneflow-Inc/oneflow/pull/5505)
- [**enhancement**][**ci**][**build**][**need-simple-ci**] Fix macOS for recent changes [5705](https://github.com/Oneflow-Inc/oneflow/pull/5705)
- [**bug**][**build**] fix return type error on gcc 4.8.5 [5660](https://github.com/Oneflow-Inc/oneflow/pull/5660)
- [**enhancement**][**build**] Check CMAKE_BUILD_TYPE [5656](https://github.com/Oneflow-Inc/oneflow/pull/5656)
- [**enhancement**][**build**] add -Werror=return-type [5655](https://github.com/Oneflow-Inc/oneflow/pull/5655)
- [**enhancement**][**build**] Clean and fix for new py dir [5618](https://github.com/Oneflow-Inc/oneflow/pull/5618)
- [**enhancement**][**build**] cmake: disable array-bounds check & treat warnings as errors for pyextobj and oneflow_internal & fix warnings [5838](https://github.com/Oneflow-Inc/oneflow/pull/5838)
- [**bug**][**build**] set CMAKE_BUILD_TYPE to Release if undefined [5842](https://github.com/Oneflow-Inc/oneflow/pull/5842)
- [**enhancement**][**build**][**need-simple-ci**] Fix all warnings & Add option TREAT_WARING_AS_ERROR to cmake [5751](https://github.com/Oneflow-Inc/oneflow/pull/5751)
- [**enhancement**][**build**] add CMAKE_INTERPROCEDURAL_OPTIMIZATION in fast cmake cache [5970](https://github.com/Oneflow-Inc/oneflow/pull/5970)
- [**enhancement**][**build**] add clang tidy target [5957](https://github.com/Oneflow-Inc/oneflow/pull/5957)
- [**bug**][**build**] cmake: fix cmake cache args in opencv [5959](https://github.com/Oneflow-Inc/oneflow/pull/5959)
- [**enhancement**][**build**] Add cmake option USE_SYSTEM_NCCL [5897](https://github.com/Oneflow-Inc/oneflow/pull/5897)
- [**enhancement**][**build**] cmake: include third party headers as system headers to avoid warnings [5879](https://github.com/Oneflow-Inc/oneflow/pull/5879)
- [**enhancement**][**build**] Ignore opencv-python on machine aarch64 [5884](https://github.com/Oneflow-Inc/oneflow/pull/5884)
- [**enhancement**][**build**] enable CMake first class cuda support [5858](https://github.com/Oneflow-Inc/oneflow/pull/5858)
- [**bug**][**build**] Fix compile warning (strict-aliasing) [5872](https://github.com/Oneflow-Inc/oneflow/pull/5872)
- [**enhancement**][**bug**][**build**][**need-simple-ci**] Upgrade gtest and fix some errors raised by clang [6079](https://github.com/Oneflow-Inc/oneflow/pull/6079)
- [**bug**][**ci**][**build**] cmake: fix ninja build in CI [6072](https://github.com/Oneflow-Inc/oneflow/pull/6072)
- [**bug**][**build**] fix files not actually removed when building for multiple python versions [6060](https://github.com/Oneflow-Inc/oneflow/pull/6060)
- [**bug**][**build**][**api**] functional_api: fix build error in mac os [6010](https://github.com/Oneflow-Inc/oneflow/pull/6010)
- [**bug**][**build**][**need-simple-ci**][**need-single-client-tests**] Fix recompile from scratch [6036](https://github.com/Oneflow-Inc/oneflow/pull/6036)
- [**bug**][**build**] Turn on NVCC's warnings [6011](https://github.com/Oneflow-Inc/oneflow/pull/6011)
- [**bug**][**build**][**need-single-client-tests**] fix bundle .so of other python version [6034](https://github.com/Oneflow-Inc/oneflow/pull/6034)
- [**bug**][**ci**][**build**][**need-single-client-tests**] use copy_all_files_in_dir to replace copy_files [6033](https://github.com/Oneflow-Inc/oneflow/pull/6033)
- [**enhancement**][**build**] check compiler version in cmake [6026](https://github.com/Oneflow-Inc/oneflow/pull/6026)
- [**enhancement**][**build**] Add CUDA_NVCC_THREADS_NUMBER [6017](https://github.com/Oneflow-Inc/oneflow/pull/6017)
- [**enhancement**][**build**][**need-simple-ci**] optimize of_include_copy [5978](https://github.com/Oneflow-Inc/oneflow/pull/5978)
- [**enhancement**][**ci**][**build**][**need-single-client-tests**] CI: remove `-DTREAT_WARNINGS_AS_ERRORS=OFF` [6008](https://github.com/Oneflow-Inc/oneflow/pull/6008)
- [**enhancement**][**build**][**xla**] xrt: fix all warnings [5915](https://github.com/Oneflow-Inc/oneflow/pull/5915)
- [**enhancement**][**build**] Prevent opencv compile failure with std 17 [5997](https://github.com/Oneflow-Inc/oneflow/pull/5997)
- [**enhancement**][**build**] Use bundled cub [5998](https://github.com/Oneflow-Inc/oneflow/pull/5998)
- [**enhancement**][**ci**][**build**] update clang tidy diff warnings-as-errors option [5989](https://github.com/Oneflow-Inc/oneflow/pull/5989)
- [**enhancement**][**build**] Update run_clang_tidy.py to set return code and add warning-as-errors [5977](https://github.com/Oneflow-Inc/oneflow/pull/5977)
- [**enhancement**][**build**] check: fix clang-tidy-diff commands [5972](https://github.com/Oneflow-Inc/oneflow/pull/5972)
- [**bug**][**build**] Suppress NVCC warning 177-D [6094](https://github.com/Oneflow-Inc/oneflow/pull/6094)

XLA enhancements:

- [**bug**][**xla**] Make the blob header memory aligned. [5286](https://github.com/Oneflow-Inc/oneflow/pull/5286)

System:

- [**enhancement**][**system**] Refactor Memory Zone [5072](https://github.com/Oneflow-Inc/oneflow/pull/5072)
- [**enhancement**][**system**] Add interface InferContext::OutputTensorDesc [5219](https://github.com/Oneflow-Inc/oneflow/pull/5219)
- [**bug**][**system**] Lazy construct functor to make sure that the operators has already been registered. [5225](https://github.com/Oneflow-Inc/oneflow/pull/5225)
- [**enhancement**][**system**] Refactor infer ctx output isdynamic [5220](https://github.com/Oneflow-Inc/oneflow/pull/5220)
- [**enhancement**][**system**] Refactor infer ctx input isdynamic [5211](https://github.com/Oneflow-Inc/oneflow/pull/5211)
- [**enhancement**][**system**] Wake up the heartbeat thread immediately [5081](https://github.com/Oneflow-Inc/oneflow/pull/5081)
- [**enhancement**][**system**] Fix xla test case fail [5203](https://github.com/Oneflow-Inc/oneflow/pull/5203)
- [**enhancement**][**system**] Add interface InferContext::InputDType [5153](https://github.com/Oneflow-Inc/oneflow/pull/5153)
- [**purge**][**system**] delete const_cast in Output [5196](https://github.com/Oneflow-Inc/oneflow/pull/5196)
- [**feature**][**system**] Add hwloc for topology detection [5291](https://github.com/Oneflow-Inc/oneflow/pull/5291)
- [**enhancement**][**system**] fix registry may segment [5336](https://github.com/Oneflow-Inc/oneflow/pull/5336)
- [**enhancement**][**system**] Use functional api instead of op_expr_helper::XXXOp. [5364](https://github.com/Oneflow-Inc/oneflow/pull/5364)
- [**enhancement**][**system**] move btob to op [5274](https://github.com/Oneflow-Inc/oneflow/pull/5274)
- [**documentation**][**system**] Add Latest News section in README [5361](https://github.com/Oneflow-Inc/oneflow/pull/5361)
- [**enhancement**][**bug**][**system**] fix dropout module: return directly if not training [5346](https://github.com/Oneflow-Inc/oneflow/pull/5346)
- [**bug**][**system**] add missing JUST [5357](https://github.com/Oneflow-Inc/oneflow/pull/5357)
- [**documentation**][**system**] Add more communication outlets on README [5359](https://github.com/Oneflow-Inc/oneflow/pull/5359)
- [**enhancement**][**feature**][**system**] CommNet dynamic register memory [5281](https://github.com/Oneflow-Inc/oneflow/pull/5281)
- [**enhancement**][**system**] Use symbol device [5341](https://github.com/Oneflow-Inc/oneflow/pull/5341)
- [**enhancement**][**system**] fix multithread bug in env [5283](https://github.com/Oneflow-Inc/oneflow/pull/5283)
- [**bug**][**system**][**api**] fix bug in cfg_replacement [5335](https://github.com/Oneflow-Inc/oneflow/pull/5335)
- [**bug**][**system**] Fix create log directory thread-unsafe [5326](https://github.com/Oneflow-Inc/oneflow/pull/5326)
- [**bug**][**system**] fix_bug_in_make_parallel [5328](https://github.com/Oneflow-Inc/oneflow/pull/5328)
- [**enhancement**][**system**][**cfg**] replace train_conf, job_conf using cfg::xx [5263](https://github.com/Oneflow-Inc/oneflow/pull/5263)
- [**enhancement**][**system**][**quantization**] support tensorrt in qat [5287](https://github.com/Oneflow-Inc/oneflow/pull/5287)
- [**enhancement**][**system**][**api**] Export functional apis for oneflow.experimental. [5313](https://github.com/Oneflow-Inc/oneflow/pull/5313)
- [**enhancement**][**system**] fix bug check between cfg enum and proto enum [5285](https://github.com/Oneflow-Inc/oneflow/pull/5285)
- [**enhancement**][**system**] replace CHECK_EQ using CHECK_EQ_OR_RETURN [5279](https://github.com/Oneflow-Inc/oneflow/pull/5279)
- [**enhancement**][**system**] Refactor SbpXXX to cfg::SbpXXX [5120](https://github.com/Oneflow-Inc/oneflow/pull/5120)
- [**enhancement**][**system**][**api**] add detach for LazyMirroredtensorImpl [5270](https://github.com/Oneflow-Inc/oneflow/pull/5270)
- [**enhancement**][**system**] shorten XXIsDynamic4ArgNameAndIndex to be xxIsDynamic [5265](https://github.com/Oneflow-Inc/oneflow/pull/5265)
- [**enhancement**][**system**][**cfg**] job_config to cfg [5235](https://github.com/Oneflow-Inc/oneflow/pull/5235)
- [**feature**][**system**] Multi-Client LogicalRun degenerate to PhysicalRun [5479](https://github.com/Oneflow-Inc/oneflow/pull/5479)
- [**enhancement**][**system**] fix ConstructOp without JUST [5480](https://github.com/Oneflow-Inc/oneflow/pull/5480)
- [**enhancement**][**system**] Output arg modifier return maybe part 1 [5451](https://github.com/Oneflow-Inc/oneflow/pull/5451)
- [**feature**][**system**][**interface**] Fea/nn graph/graph build ctx [5420](https://github.com/Oneflow-Inc/oneflow/pull/5420)
- [**enhancement**][**system**] Throw exception if check failed [5457](https://github.com/Oneflow-Inc/oneflow/pull/5457)
- [**feature**][**system**] multi client launch [5372](https://github.com/Oneflow-Inc/oneflow/pull/5372)
- [**enhancement**][**system**][**api**] Optimize reduce mean [5452](https://github.com/Oneflow-Inc/oneflow/pull/5452)
- [**enhancement**][**system**] export Tensor only to python [5440](https://github.com/Oneflow-Inc/oneflow/pull/5440)
- [**enhancement**][**system**] Output arg modifier return maybe part_0 [5447](https://github.com/Oneflow-Inc/oneflow/pull/5447)
- [**enhancement**][**system**] ThreadMgr support AddPlan [5450](https://github.com/Oneflow-Inc/oneflow/pull/5450)
- [**enhancement**][**system**] Refactor infer ctx input tensordesc [5226](https://github.com/Oneflow-Inc/oneflow/pull/5226)
- [**enhancement**][**system**][**api**] instruction builder return maybe [5442](https://github.com/Oneflow-Inc/oneflow/pull/5442)
- [**feature**][**system**][**interface**] MultiClientSessionContext [5421](https://github.com/Oneflow-Inc/oneflow/pull/5421)
- [**enhancement**][**feature**][**system**] add launcher, update multi client launch and exit [5414](https://github.com/Oneflow-Inc/oneflow/pull/5414)
- [**purge**][**system**][**refactor**] Remove IOConf [5419](https://github.com/Oneflow-Inc/oneflow/pull/5419)
- [**enhancement**][**system**] Dev refine generator [5426](https://github.com/Oneflow-Inc/oneflow/pull/5426)
- [**enhancement**][**system**] Support inplace operations [5204](https://github.com/Oneflow-Inc/oneflow/pull/5204)
- [**enhancement**][**system**][**refactor**] Dev refactor generator [5397](https://github.com/Oneflow-Inc/oneflow/pull/5397)
- [**enhancement**][**system**] Add new placement init func [5408](https://github.com/Oneflow-Inc/oneflow/pull/5408)
- [**enhancement**][**system**] NNGraphIf [5387](https://github.com/Oneflow-Inc/oneflow/pull/5387)
- [**enhancement**][**system**][**refactor**] Cast explicitily in unpack call to avoid confilt with Optional. [5380](https://github.com/Oneflow-Inc/oneflow/pull/5380)
- [**enhancement**][**system**][**interface**] [Random Generator] Part2: Migrate functional dropout [5378](https://github.com/Oneflow-Inc/oneflow/pull/5378)
- [**enhancement**][**system**] replace ForeignJobInstance using JobInstance [5374](https://github.com/Oneflow-Inc/oneflow/pull/5374)
- [**enhancement**][**system**][**refactor**] Speedup reshape module by 5x. [5381](https://github.com/Oneflow-Inc/oneflow/pull/5381)
- [**feature**][**system**][**interface**] [Random Generator] Part1: Dev random generator [5360](https://github.com/Oneflow-Inc/oneflow/pull/5360)
- [**enhancement**][**system**] Add ONEFLOW_STREAM_CUDA_EVENT_FLAG_BLOCKING_SYNC [5612](https://github.com/Oneflow-Inc/oneflow/pull/5612)
- [**enhancement**][**system**] [part2]Remove singleclient outdated api [5568](https://github.com/Oneflow-Inc/oneflow/pull/5568)
- [**feature**][**system**][**interface**] nn.Graph call and launch impl [5580](https://github.com/Oneflow-Inc/oneflow/pull/5580)
- [**enhancement**][**system**] remove outdated doctest api and "experimental_api" [5564](https://github.com/Oneflow-Inc/oneflow/pull/5564)
- [**feature**][**system**][**interface**] Register ForeignCallback and Watcher in Multi-Client [5591](https://github.com/Oneflow-Inc/oneflow/pull/5591)
- [**enhancement**][**system**] [Part-1]remove outdated api and files of multi-client on master branch [5556](https://github.com/Oneflow-Inc/oneflow/pull/5556)
- [**feature**][**system**][**interface**] LazyInterpret build LocalTensor if input is local [5582](https://github.com/Oneflow-Inc/oneflow/pull/5582)
- [**enhancement**][**system**] add job_pass MultiClientAutoSourceAndSinkTick [5507](https://github.com/Oneflow-Inc/oneflow/pull/5507)
- [**feature**][**system**] Fea/nn graph/optimizer [5533](https://github.com/Oneflow-Inc/oneflow/pull/5533)
- [**feature**][**system**][**interface**] New/CloseRuntimeBuffers and RunLazyJob impl [5571](https://github.com/Oneflow-Inc/oneflow/pull/5571)
- [**feature**][**system**][**refactor**][**interface**] NNGraph interface and implement for CompileAndRuntime [5558](https://github.com/Oneflow-Inc/oneflow/pull/5558)
- [**feature**][**system**] Fea/nn graph/forward graph [5516](https://github.com/Oneflow-Inc/oneflow/pull/5516)
- [**enhancement**][**system**] Lazy job stream type [5389](https://github.com/Oneflow-Inc/oneflow/pull/5389)
- [**enhancement**][**system**] Refactor single client autotick [5506](https://github.com/Oneflow-Inc/oneflow/pull/5506)
- [**enhancement**][**system**] replace underline using dot in single client [5547](https://github.com/Oneflow-Inc/oneflow/pull/5547)
- [**bug**][**system**] fix return type [5548](https://github.com/Oneflow-Inc/oneflow/pull/5548)
- [**feature**][**system**][**interface**] LazyInterpret for UserOpExpr [5544](https://github.com/Oneflow-Inc/oneflow/pull/5544)
- [**enhancement**][**system**] Add ProfilerStart/ProfilerStop API [5542](https://github.com/Oneflow-Inc/oneflow/pull/5542)
- [**feature**][**system**][**interface**] LazyInterpreter for FetchOutputOpExpr and set op parallel_distribution [5527](https://github.com/Oneflow-Inc/oneflow/pull/5527)
- [**enhancement**][**system**] Multi client push pull [5492](https://github.com/Oneflow-Inc/oneflow/pull/5492)
- [**enhancement**][**system**] registry_callback_fn return maybe [5456](https://github.com/Oneflow-Inc/oneflow/pull/5456)
- [**enhancement**][**system**] bw_gen_fn return maybe [5455](https://github.com/Oneflow-Inc/oneflow/pull/5455)
- [**enhancement**][**system**] gen_bw_fn return maybe [5454](https://github.com/Oneflow-Inc/oneflow/pull/5454)
- [**enhancement**][**system**] Compatible single client [5417](https://github.com/Oneflow-Inc/oneflow/pull/5417)
- [**feature**][**system**][**interface**] GlobalMultiClientEnv and refine EagerExecution [5523](https://github.com/Oneflow-Inc/oneflow/pull/5523)
- [**enhancement**][**system**] Job pass maybe system [5503](https://github.com/Oneflow-Inc/oneflow/pull/5503)
- [**enhancement**][**system**] Remove Plan::net_topo [5502](https://github.com/Oneflow-Inc/oneflow/pull/5502)
- [**feature**][**system**][**interface**] LazyInterpret for FeedVariableOpExpr [5490](https://github.com/Oneflow-Inc/oneflow/pull/5490)
- [**enhancement**][**system**] Input arg modifier return maybe [5453](https://github.com/Oneflow-Inc/oneflow/pull/5453)
- [**feature**][**system**][**interface**] Fea/nn graph/block scope [5498](https://github.com/Oneflow-Inc/oneflow/pull/5498)
- [**feature**][**system**] jit_fuse_cast_scale [5332](https://github.com/Oneflow-Inc/oneflow/pull/5332)
- [**enhancement**][**system**] Remove obsolete Profiler [5747](https://github.com/Oneflow-Inc/oneflow/pull/5747)
- [**enhancement**][**system**][**api**] Dev fix batch norm not stats [5733](https://github.com/Oneflow-Inc/oneflow/pull/5733)
- [**enhancement**][**system**] rename rpc_token to TransportToken [5735](https://github.com/Oneflow-Inc/oneflow/pull/5735)
- [**enhancement**][**system**][**api**] Refacotr maximum minimum py2cpp [5724](https://github.com/Oneflow-Inc/oneflow/pull/5724)
- [**enhancement**][**system**] Replace piece_id with comm_net_sequence_number [5731](https://github.com/Oneflow-Inc/oneflow/pull/5731)
- [**enhancement**][**system**] beautify stack frame [5686](https://github.com/Oneflow-Inc/oneflow/pull/5686)
- [**enhancement**][**system**] Add env ONEFLOW_KERNEL_DISABLE_BLOB_ACCESS_CHECKER [5728](https://github.com/Oneflow-Inc/oneflow/pull/5728)
- [**enhancement**][**system**] Add env ONEFLOW_THREAD_ENABLE_LOCAL_MESSAGE_QUEUE [5720](https://github.com/Oneflow-Inc/oneflow/pull/5720)
- [**enhancement**][**system**][**api**][**refactor**] Refactor functional sub, mul and div apis [5713](https://github.com/Oneflow-Inc/oneflow/pull/5713)
- [**feature**][**system**] ddp [5008](https://github.com/Oneflow-Inc/oneflow/pull/5008)
- [**enhancement**][**system**][**api**][**refactor**] Refactor functional matmul and add apis. [5697](https://github.com/Oneflow-Inc/oneflow/pull/5697)
- [**bug**][**system**] Fix ClearKV("plan") [5710](https://github.com/Oneflow-Inc/oneflow/pull/5710)
- [**enhancement**][**system**] Rename cpu to async cpu [5712](https://github.com/Oneflow-Inc/oneflow/pull/5712)
- [**enhancement**][**system**] Support tensor.to()/to_local() [5271](https://github.com/Oneflow-Inc/oneflow/pull/5271)
- [**feature**][**system**][**refactor**][**interface**] Multi-Runtime for multi nn.Graph [5683](https://github.com/Oneflow-Inc/oneflow/pull/5683)
- [**bug**][**system**][**refactor**] Add tag for Optional inplace constructor [5619](https://github.com/Oneflow-Inc/oneflow/pull/5619)
- [**enhancement**][**system**] Move Global<CommNet> to env scope [5670](https://github.com/Oneflow-Inc/oneflow/pull/5670)
- [**enhancement**][**system**] add JUST wrapper [5681](https://github.com/Oneflow-Inc/oneflow/pull/5681)
- [**enhancement**][**system**] New sync consistent meta info [5634](https://github.com/Oneflow-Inc/oneflow/pull/5634)
- [**enhancement**][**system**][**refactor**][**interface**] Refactor RuntimeCtx for multi-runtime [5664](https://github.com/Oneflow-Inc/oneflow/pull/5664)
- [**feature**][**system**][**interface**] Feat: memory shared between EagerTensor with VariableRegst [5649](https://github.com/Oneflow-Inc/oneflow/pull/5649)
- [**enhancement**][**system**] Use functional call directly instead of construct a module and then call-Add [5613](https://github.com/Oneflow-Inc/oneflow/pull/5613)
- [**enhancement**][**system**] disable eager_op consistent mode [5647](https://github.com/Oneflow-Inc/oneflow/pull/5647)
- [**enhancement**][**system**] add msg_penddin_list in ibverbs_qp to optimize qp_init_attr.cap.max_send_wr [5485](https://github.com/Oneflow-Inc/oneflow/pull/5485)
- [**enhancement**][**system**] IBVerbsCommNet add knobs [5626](https://github.com/Oneflow-Inc/oneflow/pull/5626)
- [**enhancement**][**system**] Prune python tensor [5596](https://github.com/Oneflow-Inc/oneflow/pull/5596)
- [**feature**][**system**][**interface**] Feat: LazyInterpret infer op / tensor ParallelDescScope [5625](https://github.com/Oneflow-Inc/oneflow/pull/5625)
- [**enhancement**][**system**] Replace src tick with with wait and send ids [5603](https://github.com/Oneflow-Inc/oneflow/pull/5603)
- [**enhancement**][**system**] Support symbol placement type in functional. [5627](https://github.com/Oneflow-Inc/oneflow/pull/5627)
- [**enhancement**][**system**][**api**][**refactor**][**interface**] Dev advanced indexing [5559](https://github.com/Oneflow-Inc/oneflow/pull/5559)
- [**enhancement**][**system**] Optimize maybe. [5839](https://github.com/Oneflow-Inc/oneflow/pull/5839)
- [**enhancement**][**system**] Decorator 4 disable recursive boxing call [5796](https://github.com/Oneflow-Inc/oneflow/pull/5796)
- [**enhancement**][**system**] add_eager_boxing_and_op_interpreter_dispatch_error_info [5819](https://github.com/Oneflow-Inc/oneflow/pull/5819)
- [**enhancement**][**system**] Kernel CUDA Graphs Support [5725](https://github.com/Oneflow-Inc/oneflow/pull/5725)
- [**bug**][**system**] Fix placement print bug [5853](https://github.com/Oneflow-Inc/oneflow/pull/5853)
- [**bug**][**system**] when error msg formatting fails, return error->DebugString [5844](https://github.com/Oneflow-Inc/oneflow/pull/5844)
- [**enhancement**][**system**][**refactor**] Rename variables named `*parallel_distribution*` to `*nd_sbp*` (1) [5815](https://github.com/Oneflow-Inc/oneflow/pull/5815)
- [**feature**][**system**][**interface**] Support Free EagerTensor caught in nn.Graph build [5777](https://github.com/Oneflow-Inc/oneflow/pull/5777)
- [**enhancement**][**system**] Reuse CUDA event / Refine BnInOp2Blob / Refine channel [5837](https://github.com/Oneflow-Inc/oneflow/pull/5837)
- [**enhancement**][**system**][**serving**] fix bug in AddInputOutputOpsPass: check existence of key in HashMap(inferface_lbi2scope_sym_id) [5653](https://github.com/Oneflow-Inc/oneflow/pull/5653)
- [**enhancement**][**system**][**api**] unpack_call: impl new `unpack_call_dispatcher` for better performance [5820](https://github.com/Oneflow-Inc/oneflow/pull/5820)
- [**feature**][**system**] Feat consistent tensor python constructor [5812](https://github.com/Oneflow-Inc/oneflow/pull/5812)
- [**feature**][**system**] Support 0shape tensor [5620](https://github.com/Oneflow-Inc/oneflow/pull/5620)
- [**documentation**][**system**] fix launcher description [5770](https://github.com/Oneflow-Inc/oneflow/pull/5770)
- [**feature**][**system**][**interface**] Multi-nn.Graph memory reuse by Chunk manager [5658](https://github.com/Oneflow-Inc/oneflow/pull/5658)
- [**bug**][**system**] Fix naive b2p error [5806](https://github.com/Oneflow-Inc/oneflow/pull/5806)
- [**enhancement**][**system**] set created generator with default rng seed [5801](https://github.com/Oneflow-Inc/oneflow/pull/5801)
- [**enhancement**][**system**] enhance_local_to_consistent [5761](https://github.com/Oneflow-Inc/oneflow/pull/5761)
- [**feature**][**system**] add flow.randn [5736](https://github.com/Oneflow-Inc/oneflow/pull/5736)
- [**enhancement**][**system**] Refactor hierarchical parallel cast autograd [5764](https://github.com/Oneflow-Inc/oneflow/pull/5764)
- [**enhancement**][**system**] Collective boxing executor add_plan delete_plan [5495](https://github.com/Oneflow-Inc/oneflow/pull/5495)
- [**enhancement**][**system**] Fix throw abort [5795](https://github.com/Oneflow-Inc/oneflow/pull/5795)
- [**enhancement**][**system**] DECORATE [5794](https://github.com/Oneflow-Inc/oneflow/pull/5794)
- [**enhancement**][**system**] Inferface eager boxing [5682](https://github.com/Oneflow-Inc/oneflow/pull/5682)
- [**enhancement**][**system**] extract_consistent_to_consistent_op_expr [5870](https://github.com/Oneflow-Inc/oneflow/pull/5870)
- [**enhancement**][**system**] disable backward pass consistent tensor meta check. [5871](https://github.com/Oneflow-Inc/oneflow/pull/5871)
- [**enhancement**][**system**] Add CudaStreamIndexGenerator::GenerateNamedStreamIndex [5940](https://github.com/Oneflow-Inc/oneflow/pull/5940)
- [**bug**][**system**] Only query PCI bus id when CUDA version >= 11 [5937](https://github.com/Oneflow-Inc/oneflow/pull/5937)
- [**enhancement**][**system**] maybe: add `JUST_MSG` and `CHECK_JUST_MSG` [5904](https://github.com/Oneflow-Inc/oneflow/pull/5904)
- [**bug**][**system**] Fix bug scalar [5950](https://github.com/Oneflow-Inc/oneflow/pull/5950)
- [**enhancement**][**system**] framework: fix rvalue reference warnings [5948](https://github.com/Oneflow-Inc/oneflow/pull/5948)
- [**purge**][**system**] Remove CudaWorkType [5942](https://github.com/Oneflow-Inc/oneflow/pull/5942)
- [**enhancement**][**system**] refactor_symbol [5941](https://github.com/Oneflow-Inc/oneflow/pull/5941)
- [**bug**][**system**] consistent_tensor_infer_cache: fix memory leak [5938](https://github.com/Oneflow-Inc/oneflow/pull/5938)
- [**feature**][**system**] support to print gpu [5936](https://github.com/Oneflow-Inc/oneflow/pull/5936)
- [**enhancement**][**system**] Bugfix static check [5935](https://github.com/Oneflow-Inc/oneflow/pull/5935)
- [**bug**][**system**] fix nccl_version log [5934](https://github.com/Oneflow-Inc/oneflow/pull/5934)
- [**bug**][**system**] Fix bug of multi-GPU train nn.Graph extra mem cost in rank 0 [5930](https://github.com/Oneflow-Inc/oneflow/pull/5930)
- [**enhancement**][**system**] Only gradient acc be scheduled in parallel. [5926](https://github.com/Oneflow-Inc/oneflow/pull/5926)
- [**enhancement**][**bug**][**system**] fix_ddp_bug_on_8_process [5929](https://github.com/Oneflow-Inc/oneflow/pull/5929)
- [**enhancement**][**system**] Fix bug error msg format [5866](https://github.com/Oneflow-Inc/oneflow/pull/5866)
- [**feature**][**system**] print consistent tensor data [5902](https://github.com/Oneflow-Inc/oneflow/pull/5902)
- [**bug**][**system**] Move parse env to the constructor [5922](https://github.com/Oneflow-Inc/oneflow/pull/5922)
- [**enhancement**][**system**] Remove GlobalWorkStreamId/GlobalThrdId [5917](https://github.com/Oneflow-Inc/oneflow/pull/5917)
- [**bug**][**system**] shared_or_scalar: fix alias warnings [5916](https://github.com/Oneflow-Inc/oneflow/pull/5916)
- [**purge**][**system**] Remove CompActor [5919](https://github.com/Oneflow-Inc/oneflow/pull/5919)
- [**enhancement**][**system**] Use symbol dtype [5641](https://github.com/Oneflow-Inc/oneflow/pull/5641)
- [**enhancement**][**feature**][**system**] Control Graph / Session / Env's python c++ object destruction [5845](https://github.com/Oneflow-Inc/oneflow/pull/5845)
- [**enhancement**][**bug**][**system**] Sync access and assign indexing tensor. [5907](https://github.com/Oneflow-Inc/oneflow/pull/5907)
- [**enhancement**][**system**][**api**][**refactor**] Dev consistent arange [5883](https://github.com/Oneflow-Inc/oneflow/pull/5883)
- [**enhancement**][**system**] Lazy interpreter for new ConsistentToConsistentOpExpr [5903](https://github.com/Oneflow-Inc/oneflow/pull/5903)
- [**bug**][**system**] Fix BUG of LazyInterpret FreeEagerTensor memory shared with regst [5891](https://github.com/Oneflow-Inc/oneflow/pull/5891)
- [**bug**][**system**] fix typo in `raise RuntimeError` [5890](https://github.com/Oneflow-Inc/oneflow/pull/5890)
- [**enhancement**][**system**][**refactor**] Rename the `ParallelDistribution` class to `NdSbp` [5814](https://github.com/Oneflow-Inc/oneflow/pull/5814)
- [**feature**][**system**] add flow.rand [5722](https://github.com/Oneflow-Inc/oneflow/pull/5722)
- [**feature**][**system**] Lazy Interpret support infer default device cpu [5880](https://github.com/Oneflow-Inc/oneflow/pull/5880)
- [**enhancement**][**system**] Tensor str [5783](https://github.com/Oneflow-Inc/oneflow/pull/5783)
- [**feature**][**system**][**interface**] Lazy to_consistent [5774](https://github.com/Oneflow-Inc/oneflow/pull/5774)
- [**enhancement**][**system**] wait vm empty before exiting [5860](https://github.com/Oneflow-Inc/oneflow/pull/5860)
- [**enhancement**][**system**] Eager boxing n to 1 [5949](https://github.com/Oneflow-Inc/oneflow/pull/5949)
- [**enhancement**][**system**] add kernel observer [6052](https://github.com/Oneflow-Inc/oneflow/pull/6052)
- [**enhancement**][**ci**][**system**] Optimize ddp broadcast and add speed/memory test in ci [6044](https://github.com/Oneflow-Inc/oneflow/pull/6044)
- [**enhancement**][**system**] add var to control only print warning once when blocked [6045](https://github.com/Oneflow-Inc/oneflow/pull/6045)
- [**enhancement**][**system**][**refactor**] Rewrite pow and logical functional apis [6032](https://github.com/Oneflow-Inc/oneflow/pull/6032)
- [**enhancement**][**system**] Token seq id [5964](https://github.com/Oneflow-Inc/oneflow/pull/5964)
- [**enhancement**][**documentation**][**system**] Remove python function wrapper. [6012](https://github.com/Oneflow-Inc/oneflow/pull/6012)
- [**feature**][**system**] Add timeout and loc for blocking calls [6007](https://github.com/Oneflow-Inc/oneflow/pull/6007)
- [**enhancement**][**system**] Eager boxing 1 to n [5943](https://github.com/Oneflow-Inc/oneflow/pull/5943)
- [**enhancement**][**system**] Boxing expr [6015](https://github.com/Oneflow-Inc/oneflow/pull/6015)
- [**enhancement**][**system**] new_X_to_B [5987](https://github.com/Oneflow-Inc/oneflow/pull/5987)
- [**enhancement**][**system**] Add unimplemented return information [5952](https://github.com/Oneflow-Inc/oneflow/pull/5952)
- [**enhancement**][**system**] Revert "Faster decorator" [6006](https://github.com/Oneflow-Inc/oneflow/pull/6006)
- [**enhancement**][**system**] Throw exception if using advanced indexing for tensor setitem [6001](https://github.com/Oneflow-Inc/oneflow/pull/6001)
- [**enhancement**][**system**] Support eager boxing sm 2 sn [5869](https://github.com/Oneflow-Inc/oneflow/pull/5869)
- [**enhancement**][**system**] Move framework/local_dep_object.\* to the eager directory [5988](https://github.com/Oneflow-Inc/oneflow/pull/5988)
- [**enhancement**][**system**] Fix builtin op arg tuple. [5464](https://github.com/Oneflow-Inc/oneflow/pull/5464)
- [**feature**][**system**][**refactor**] Dev functional multiple signatures [5982](https://github.com/Oneflow-Inc/oneflow/pull/5982)
- [**enhancement**][**system**] Faster decorator [5996](https://github.com/Oneflow-Inc/oneflow/pull/5996)
- [**enhancement**][**system**] Placed nd sbp [5995](https://github.com/Oneflow-Inc/oneflow/pull/5995)
- [**feature**][**system**] Support asymmetric input/output/variable tensors in nn.Graph [5983](https://github.com/Oneflow-Inc/oneflow/pull/5983)
- [**enhancement**][**system**] LightActor [5868](https://github.com/Oneflow-Inc/oneflow/pull/5868)
- [**bug**][**system**] Prevent running oneflow in forked subprocess [5976](https://github.com/Oneflow-Inc/oneflow/pull/5976)
- [**bug**][**system**] common/error: fix build error in mac os [5971](https://github.com/Oneflow-Inc/oneflow/pull/5971)
- [**bug**][**system**] fix_bug_test_tensor_str [5958](https://github.com/Oneflow-Inc/oneflow/pull/5958)
- [**enhancement**][**system**] Refine StreamContext [6191](https://github.com/Oneflow-Inc/oneflow/pull/6191)
- [**enhancement**][**system**] container_util: fix VectorAt, remove useless MutMapAt [6172](https://github.com/Oneflow-Inc/oneflow/pull/6172)
- [**enhancement**][**system**] Typesafe KernelState [6198](https://github.com/Oneflow-Inc/oneflow/pull/6198)
- [**enhancement**][**system**] Primitive based copy task node [6195](https://github.com/Oneflow-Inc/oneflow/pull/6195)
- [**feature**][**system**][**interface**] Lazy support Scalar [6181](https://github.com/Oneflow-Inc/oneflow/pull/6181)
- [**enhancement**][**system**] Disable implicit boxing when parallel num eq one [6188](https://github.com/Oneflow-Inc/oneflow/pull/6188)
- [**enhancement**][**system**] Primitive [6183](https://github.com/Oneflow-Inc/oneflow/pull/6183)
- [**enhancement**][**system**] Remove IDMgr::GetGpuPhyIdFromThrdId/IDMgr::GetDeviceTypeFromThrdId [6169](https://github.com/Oneflow-Inc/oneflow/pull/6169)
- [**enhancement**][**system**] remove op_expr_helper inside gradient_funcs [6057](https://github.com/Oneflow-Inc/oneflow/pull/6057)
- [**feature**][**system**][**api**] Add tensor yaml, support export tensor functional api. [6099](https://github.com/Oneflow-Inc/oneflow/pull/6099)
- [**feature**][**system**] Plan memory log [6151](https://github.com/Oneflow-Inc/oneflow/pull/6151)
- [**feature**][**system**] Add dtype bfloat16 [5304](https://github.com/Oneflow-Inc/oneflow/pull/5304)
- [**enhancement**][**system**] StreamContext [6129](https://github.com/Oneflow-Inc/oneflow/pull/6129)
- [**bug**][**system**] Fix wrong inplace acc grad [6146](https://github.com/Oneflow-Inc/oneflow/pull/6146)
- [**enhancement**][**system**] UserKernel remove job_desc [6144](https://github.com/Oneflow-Inc/oneflow/pull/6144)
- [**enhancement**][**system**][**api**] Fea/graph/add outputs buffer to enable pipeline [6126](https://github.com/Oneflow-Inc/oneflow/pull/6126)
- [**enhancement**][**system**] not fuse request for nccl 2.10.3 [6136](https://github.com/Oneflow-Inc/oneflow/pull/6136)
- [**bug**][**system**] NewUniqueId thread safe [6141](https://github.com/Oneflow-Inc/oneflow/pull/6141)
- [**enhancement**][**system**] XRT remove job_desc [6139](https://github.com/Oneflow-Inc/oneflow/pull/6139)
- [**enhancement**][**system**] SystemOpFillJobNamePass [6138](https://github.com/Oneflow-Inc/oneflow/pull/6138)
- [**enhancement**][**system**] mv_boxing_folder_to_core [6140](https://github.com/Oneflow-Inc/oneflow/pull/6140)
- [**enhancement**][**system**] Refactor boxing interpreter to boxing expr [6134](https://github.com/Oneflow-Inc/oneflow/pull/6134)
- [**enhancement**][**system**] Eager boxing one to one [6048](https://github.com/Oneflow-Inc/oneflow/pull/6048)
- [**enhancement**][**system**] Vm cpu efficiency [6110](https://github.com/Oneflow-Inc/oneflow/pull/6110)
- [**enhancement**][**system**] Naive generic boxing [6116](https://github.com/Oneflow-Inc/oneflow/pull/6116)
- [**feature**][**system**] send/recv [5992](https://github.com/Oneflow-Inc/oneflow/pull/5992)
- [**enhancement**][**system**] disable_print_stack_in_tensor_numpy [6123](https://github.com/Oneflow-Inc/oneflow/pull/6123)
- [**feature**][**system**] add all_reduce by to_consistent [5963](https://github.com/Oneflow-Inc/oneflow/pull/5963)
- [**enhancement**][**system**] KernelContext [6084](https://github.com/Oneflow-Inc/oneflow/pull/6084)
- [**enhancement**][**bug**][**system**] Fix sync nccl and async nccl deadlock [6071](https://github.com/Oneflow-Inc/oneflow/pull/6071)
- [**bug**][**system**][**refactor**] Refactor to local [6098](https://github.com/Oneflow-Inc/oneflow/pull/6098)
- [**enhancement**][**system**] Replace xor with hash combine (part 1) [6078](https://github.com/Oneflow-Inc/oneflow/pull/6078)
- [**enhancement**][**system**] Optimize error message [6073](https://github.com/Oneflow-Inc/oneflow/pull/6073)
- [**enhancement**][**system**] Rename Error::xx to Error::xxError [6049](https://github.com/Oneflow-Inc/oneflow/pull/6049)
- [**enhancement**][**system**] send formatted msg to glog [5999](https://github.com/Oneflow-Inc/oneflow/pull/5999)
- [**feature**][**bottleneck**][**bug**][**system**][**interface**] [Feat.] NNGraph new eager tensor for new variable created in JobPass [6091](https://github.com/Oneflow-Inc/oneflow/pull/6091)
- [**bug**][**system**] Fix bug of multi-GPU eager copy D2H extra mem cost in rank 0 [6092](https://github.com/Oneflow-Inc/oneflow/pull/6092)
- [**enhancement**][**system**][**api**] Rename module flow.F to flow.\_C [6053](https://github.com/Oneflow-Inc/oneflow/pull/6053)
- [**feature**][**system**][**interface**] [Feat.] Eager consistent OFRecordReader [6089](https://github.com/Oneflow-Inc/oneflow/pull/6089)
- [**enhancement**][**system**][**api**] Dev fix and align interface [6075](https://github.com/Oneflow-Inc/oneflow/pull/6075)
- [**feature**][**bottleneck**][**bug**][**system**][**interface**] NNGraph input/output valid by register tensors [6240](https://github.com/Oneflow-Inc/oneflow/pull/6240)
- [**bug**][**system**][**interface**] Fix bug of Multi-Client src tick output order [6221](https://github.com/Oneflow-Inc/oneflow/pull/6221)
- [**enhancement**][**bug**][**system**] Add cast primitive [6234](https://github.com/Oneflow-Inc/oneflow/pull/6234)
- [**feature**][**bottleneck**][**system**][**interface**] Auto FixPipelineStageIdPass [6204](https://github.com/Oneflow-Inc/oneflow/pull/6204)
- [**enhancement**][**system**] move scalar to oneflow namespace. [6235](https://github.com/Oneflow-Inc/oneflow/pull/6235)
- [**enhancement**][**system**] UserKernel init CUDA Graphs with state [6230](https://github.com/Oneflow-Inc/oneflow/pull/6230)
- [**feature**][**system**] Comm broadcast [6213](https://github.com/Oneflow-Inc/oneflow/pull/6213)
- [**enhancement**][**system**][**refactor**] Rename op*name* to op*type_name* in AutogradEngine [6154](https://github.com/Oneflow-Inc/oneflow/pull/6154)
- [**enhancement**][**system**] Add memset primitive [6218](https://github.com/Oneflow-Inc/oneflow/pull/6218)
- [**enhancement**][**system**] Add StreamContext::device_type()/DeviceCtx::device_type() [6217](https://github.com/Oneflow-Inc/oneflow/pull/6217)
- [**feature**][**system**] add all_gather and fix bug of multi rank doctest [6189](https://github.com/Oneflow-Inc/oneflow/pull/6189)
- [**feature**][**system**][**interface**] [Feat.] Lazy interpreter skip hierarchical_parallel_cast [6208](https://github.com/Oneflow-Inc/oneflow/pull/6208)
- [**purge**][**system**] Cleanup KernelUtil [6212](https://github.com/Oneflow-Inc/oneflow/pull/6212)
- [**enhancement**][**system**] StreamContextAdapter [6205](https://github.com/Oneflow-Inc/oneflow/pull/6205)
- [**enhancement**][**system**] Dev eliminate gcc warnings [6199](https://github.com/Oneflow-Inc/oneflow/pull/6199)
- [**feature**][**bottleneck**][**system**][**interface**] [Feat.] nn.Graph support grad acc with input/output tensor [6155](https://github.com/Oneflow-Inc/oneflow/pull/6155)
- [**enhancement**][**system**] Cpu symetric s to s [6153](https://github.com/Oneflow-Inc/oneflow/pull/6153)
- [**enhancement**][**system**][**upload-core**] Op expr infer tensor meta [5064](https://github.com/Oneflow-Inc/oneflow/pull/5064)
- [**enhancement**][**system**] Infer consistent tensor meta [5362](https://github.com/Oneflow-Inc/oneflow/pull/5362)

CI enhancements:

- [**bug**][**ci**][**api**][**interface**] Refine module test [5232](https://github.com/Oneflow-Inc/oneflow/pull/5232)
- [**enhancement**][**ci**] Add Simple CI, runs CPU-only on GitHub hosted servers [5207](https://github.com/Oneflow-Inc/oneflow/pull/5207)
- [**enhancement**][**ci**] Run exe test in CPU-only [5202](https://github.com/Oneflow-Inc/oneflow/pull/5202)
- [**enhancement**][**ci**] Cancel all workflow runs but the latest [5206](https://github.com/Oneflow-Inc/oneflow/pull/5206)
- [**enhancement**][**ci**] Fix master not running Simple CI [5368](https://github.com/Oneflow-Inc/oneflow/pull/5368)
- [**enhancement**][**ci**] Refine Simple CI and Clang analysis [5367](https://github.com/Oneflow-Inc/oneflow/pull/5367)
- [**enhancement**][**feature**][**bug**][**ci**][**documentation**][**interface**] Fix upsample bilinear bug [5363](https://github.com/Oneflow-Inc/oneflow/pull/5363)
- [**enhancement**][**ci**] Build nightly for py39 [5318](https://github.com/Oneflow-Inc/oneflow/pull/5318)
- [**enhancement**][**ci**] Try distributed run for 3 times to prevent failure [5305](https://github.com/Oneflow-Inc/oneflow/pull/5305)
- [**enhancement**][**ci**] Upload Simple CI logs to cloud [5268](https://github.com/Oneflow-Inc/oneflow/pull/5268)
- [**enhancement**][**ci**] Remove cpu_op_eager and cuda_op_eager [5470](https://github.com/Oneflow-Inc/oneflow/pull/5470)
- [**bug**][**ci**] fix segfault in clang plugin [5437](https://github.com/Oneflow-Inc/oneflow/pull/5437)
- [**enhancement**][**ci**] Refine Simple CI error output [5435](https://github.com/Oneflow-Inc/oneflow/pull/5435)
- [**enhancement**][**ci**] Add conda env to Simple CI [5385](https://github.com/Oneflow-Inc/oneflow/pull/5385)
- [**enhancement**][**ci**] Fix clang plugin core file not found [5390](https://github.com/Oneflow-Inc/oneflow/pull/5390)
- [**bug**][**ci**] upload core when build with clang plugin [5384](https://github.com/Oneflow-Inc/oneflow/pull/5384)
- [**bug**][**ci**] clang plugin skip more files [5373](https://github.com/Oneflow-Inc/oneflow/pull/5373)
- [**enhancement**][**ci**] Use gh-action-scheduler-v2 [5370](https://github.com/Oneflow-Inc/oneflow/pull/5370)
- [**enhancement**][**ci**] relax speed threshold [5569](https://github.com/Oneflow-Inc/oneflow/pull/5569)
- [**bug**][**ci**] Fix wrong test path under compatible [5567](https://github.com/Oneflow-Inc/oneflow/pull/5567)
- [**enhancement**][**ci**][**need-simple-ci**] Prevent upload logs automatically [5560](https://github.com/Oneflow-Inc/oneflow/pull/5560)
- [**enhancement**][**ci**][**interface**] Add `nn.AdaptiveAvgPool1d` and `nn.AdaptiveAvgPool3d` [5445](https://github.com/Oneflow-Inc/oneflow/pull/5445)
- [**feature**][**ci**] add speed test in ci [5496](https://github.com/Oneflow-Inc/oneflow/pull/5496)
- [**enhancement**][**ci**] Reduce usage of Simple CI [5546](https://github.com/Oneflow-Inc/oneflow/pull/5546)
- [**feature**][**bug**][**ci**][**api**] Restruct upsample module [5524](https://github.com/Oneflow-Inc/oneflow/pull/5524)
- [**feature**][**ci**] multi client launcher test [5488](https://github.com/Oneflow-Inc/oneflow/pull/5488)
- [**enhancement**][**ci**] Remove automerge if cuda_new_interface failed [5519](https://github.com/Oneflow-Inc/oneflow/pull/5519)
- [**enhancement**][**ci**] Prevent adding subdir in python/test [5514](https://github.com/Oneflow-Inc/oneflow/pull/5514)
- [**enhancement**][**ci**] piprepo->pipindex [5517](https://github.com/Oneflow-Inc/oneflow/pull/5517)
- [**enhancement**][**ci**] add dynamic_loss_scale in ci tests [5337](https://github.com/Oneflow-Inc/oneflow/pull/5337)
- [**enhancement**][**ci**] Add timeout for wait_gpu_slot [5497](https://github.com/Oneflow-Inc/oneflow/pull/5497)
- [**enhancement**][**feature**][**ci**] new static check based on clang-tidy [5476](https://github.com/Oneflow-Inc/oneflow/pull/5476)
- [**enhancement**][**ci**] Fix url not downloadable in some browers [5701](https://github.com/Oneflow-Inc/oneflow/pull/5701)
- [**feature**][**ci**] multi client multi machine test [5685](https://github.com/Oneflow-Inc/oneflow/pull/5685)
- [**enhancement**][**ci**] Add cpu new interface CI [5639](https://github.com/Oneflow-Inc/oneflow/pull/5639)
- [**enhancement**][**ci**][**need-simple-ci**] Mv clangtidy to simple ci [5667](https://github.com/Oneflow-Inc/oneflow/pull/5667)
- [**enhancement**][**ci**][**need-simple-ci**] use clang tidy appimage in ci [5841](https://github.com/Oneflow-Inc/oneflow/pull/5841)
- [**enhancement**][**ci**] Use gcc 7 in release to prevent error [5840](https://github.com/Oneflow-Inc/oneflow/pull/5840)
- [**enhancement**][**ci**] bn tol 1e-4 => 1e-3 [5811](https://github.com/Oneflow-Inc/oneflow/pull/5811)
- [**enhancement**][**ci**] fix distributed run on built dir [5810](https://github.com/Oneflow-Inc/oneflow/pull/5810)
- [**enhancement**][**ci**] fix third party mirror check_sum [5802](https://github.com/Oneflow-Inc/oneflow/pull/5802)
- [**ci**][**documentation**] find more accurately which files need to be doctested [5782](https://github.com/Oneflow-Inc/oneflow/pull/5782)
- [**enhancement**][**ci**] Print stack unconditionally [5779](https://github.com/Oneflow-Inc/oneflow/pull/5779)
- [**enhancement**][**ci**][**need-simple-ci**] Enable more checkers for clang-tidy in CI [5738](https://github.com/Oneflow-Inc/oneflow/pull/5738)
- [**enhancement**][**ci**] CI: add clang-tidy check to test.yaml [5920](https://github.com/Oneflow-Inc/oneflow/pull/5920)
- [**ci**][**documentation**] fix docstring in oneflow.nn.functional namespace [5807](https://github.com/Oneflow-Inc/oneflow/pull/5807)
- [**enhancement**][**ci**] disable TREAT_WARNINGS_AS_ERRORS in Release CI [5886](https://github.com/Oneflow-Inc/oneflow/pull/5886)
- [**enhancement**][**ci**] Skip ci jobs by git diff [5863](https://github.com/Oneflow-Inc/oneflow/pull/5863)
- [**bug**][**ci**] quick fix 5978 [6030](https://github.com/Oneflow-Inc/oneflow/pull/6030)
- [**enhancement**][**bug**][**ci**] fix clang tidy diff options and file format [5990](https://github.com/Oneflow-Inc/oneflow/pull/5990)
- [**enhancement**][**ci**] add flow.relu [5847](https://github.com/Oneflow-Inc/oneflow/pull/5847)
- [**enhancement**][**ci**] equal => allclose [6164](https://github.com/Oneflow-Inc/oneflow/pull/6164)
- [**bug**][**ci**][**need-simple-ci**] CI: fix clang tidy checks in simple ci [6161](https://github.com/Oneflow-Inc/oneflow/pull/6161)
- [**enhancement**][**bug**][**ci**][**documentation**][**api**] add interpolate and layer_norm docs [6157](https://github.com/Oneflow-Inc/oneflow/pull/6157)
- [**bug**][**ci**] update speed test [6113](https://github.com/Oneflow-Inc/oneflow/pull/6113)
- [**enhancement**][**bug**][**ci**][**documentation**][**api**] speed import oneflow [6107](https://github.com/Oneflow-Inc/oneflow/pull/6107)
- [**bug**][**ci**] Also try install dev deps for CODEGEN_PYTHON_EXECUTABLE [6115](https://github.com/Oneflow-Inc/oneflow/pull/6115)
- [**bug**][**ci**][**need-simple-ci**] set gtest_CMAKE_DEBUG_POSTFIX "d" [6085](https://github.com/Oneflow-Inc/oneflow/pull/6085)
- [**enhancement**][**ci**] add cache init file for clang and CI build with clang [6062](https://github.com/Oneflow-Inc/oneflow/pull/6062)
- [**enhancement**][**ci**] add emoji in speed test output, make it continue-on-error [6214](https://github.com/Oneflow-Inc/oneflow/pull/6214)

Test enhancements:

- [**bug**][**test**][**interface**] Fix acos ci bug [5217](https://github.com/Oneflow-Inc/oneflow/pull/5217)
- [**feature**][**test**] implement automated test [5321](https://github.com/Oneflow-Inc/oneflow/pull/5321)
- [**enhancement**][**test**] move generator test into ops folder to accelerate tests [5472](https://github.com/Oneflow-Inc/oneflow/pull/5472)
- [**feature**][**test**][**api**] Add autotest part2 [5467](https://github.com/Oneflow-Inc/oneflow/pull/5467)
- [**enhancement**][**test**][**api**][**interface**] Add some tests with the new framework for auto testing [5561](https://github.com/Oneflow-Inc/oneflow/pull/5561)
- [**bug**][**test**] fix test error when do multi case test on graph [5590](https://github.com/Oneflow-Inc/oneflow/pull/5590)
- [**enhancement**][**test**] Refine module test using auto test by yaochi [5484](https://github.com/Oneflow-Inc/oneflow/pull/5484)
- [**enhancement**][**test**] Add autotest for BatchNorm2d [5734](https://github.com/Oneflow-Inc/oneflow/pull/5734)
- [**enhancement**][**test**] RTH_update_op_test [5823](https://github.com/Oneflow-Inc/oneflow/pull/5823)
- [**enhancement**][**test**] dev adamw graph config [5745](https://github.com/Oneflow-Inc/oneflow/pull/5745)
- [**feature**][**test**][**api**][**interface**] Add new autotest [5562](https://github.com/Oneflow-Inc/oneflow/pull/5562)
- [**bug**][**test**] restore test of alexnet graph [5798](https://github.com/Oneflow-Inc/oneflow/pull/5798)
- [**enhancement**][**test**][**interface**] add zhangshen op-test [5600](https://github.com/Oneflow-Inc/oneflow/pull/5600)
- [**feature**][**bug**][**tooling**][**test**][**interface**] Record autotest wrong code [5923](https://github.com/Oneflow-Inc/oneflow/pull/5923)
- [**enhancement**][**feature**][**test**][**api**] add randint [5718](https://github.com/Oneflow-Inc/oneflow/pull/5718)
- [**bug**][**test**] fix multi machine test [5984](https://github.com/Oneflow-Inc/oneflow/pull/5984)
- [**enhancement**][**test**][**interface**] some op test [6095](https://github.com/Oneflow-Inc/oneflow/pull/6095)

Tooling enhancements:

- [**bug**][**tooling**] user/summary: fix memory leak in `FillImageInSummary` [5742](https://github.com/Oneflow-Inc/oneflow/pull/5742)
- [**enhancement**][**tooling**][**cfg**] cfg: add move assignment operator for performance [5962](https://github.com/Oneflow-Inc/oneflow/pull/5962)
- [**enhancement**][**tooling**][**api**][**refactor**] refactor_all_device_placement_api [6080](https://github.com/Oneflow-Inc/oneflow/pull/6080)

Highlights

- First class support for eager execution. The deprecated APIs are moved to `oneflow.compatible.single_client`
- Drop-in replacement of `import torch` for existing Pytorch projects. You could test it by inter-changing `import oneflow as torch` and `import torch as flow`.
- [nn.Module](https://docs.oneflow.org/master/basics/04_build_network.html#module) for eager execution
- [nn.Graph](https://docs.oneflow.org/master/basics/08_nn_graph.html) for lazy execution
- [DDP](https://oneflow.readthedocs.io/en/master/nn.html#oneflow.nn.parallel.DistributedDataParallel) for data parallel

A sneak peek of the new API

Here is a minimum example showcasing how to incorporate a `nn.Module` in a `nn.Graph` and have it run in lazy mode.

python
class NeuralGraph(flow.nn.Graph):
def __init__(self, ...):
super().__init__()
self.model = model model is a nn.Module instance

def build(self, x):
y_pred = self.model(x)
return y_pred

graph = NeuralGraph() to create a nn.Graph instance
y_pred = graph(x) to run the created nn.Graph


New in Python API

- [**feature**][**eager**][**op**][**test**][**python**][**interface**] Add test for convtranspose2d [5239](https://github.com/Oneflow-Inc/oneflow/pull/5239)
- [**enhancement**][**python**][**interface**] Add GroupNorm [5175](https://github.com/Oneflow-Inc/oneflow/pull/5175)
- [**enhancement**][**eager**][**python**][**interface**] [Add] avgpool1d avgpool3d [5165](https://github.com/Oneflow-Inc/oneflow/pull/5165)
- [**feature**][**eager**][**op**][**python**][**interface**] Add deconv cpu impl [5224](https://github.com/Oneflow-Inc/oneflow/pull/5224)
- [**bug**][**eager**][**api**][**python**][**interface**] Fix acosh bug [5221](https://github.com/Oneflow-Inc/oneflow/pull/5221)
- [**feature**][**eager**][**op**][**python**][**interface**] Dev modules ctc loss [5168](https://github.com/Oneflow-Inc/oneflow/pull/5168)
- [**bottleneck**][**bug**][**documentation**][**python**][**interface**] Fix meshgrid test bug [5208](https://github.com/Oneflow-Inc/oneflow/pull/5208)
- [**eager**][**documentation**][**python**][**interface**] Rename CosineScheduler to CosineAnnealingLR [5112](https://github.com/Oneflow-Inc/oneflow/pull/5112)
- [**feature**][**eager**][**python**][**interface**] Add meshgrid module [5205](https://github.com/Oneflow-Inc/oneflow/pull/5205)
- [**enhancement**][**feature**][**bug**][**op**][**python**] support bias in conv2d's parameter list [5322](https://github.com/Oneflow-Inc/oneflow/pull/5322)
- [**eager**][**documentation**][**api**][**python**][**interface**] add not_equal, greater_equal and less_equal module [5350](https://github.com/Oneflow-Inc/oneflow/pull/5350)
- [**enhancement**][**eager**][**python**] refine pow module and its test [5319](https://github.com/Oneflow-Inc/oneflow/pull/5319)
- [**enhancement**][**eager**][**op**][**python**] Add triu op [5329](https://github.com/Oneflow-Inc/oneflow/pull/5329)
- [**enhancement**][**bug**][**python**] Fix optimizer for not supporting all kinds of iterables [5355](https://github.com/Oneflow-Inc/oneflow/pull/5355)
- [**bug**][**python**][**interface**] raise IndexError in get_canonical_index to support for loop [5345](https://github.com/Oneflow-Inc/oneflow/pull/5345)
- [**bug**][**python**][**interface**] tensor slice assign supports broadcasting [5344](https://github.com/Oneflow-Inc/oneflow/pull/5344)
- [**enhancement**][**op**][**python**] add cpu group conv logic [5314](https://github.com/Oneflow-Inc/oneflow/pull/5314)
- [**enhancement**][**python**] Add 'nn.Mish' module and corresponding functions [5310](https://github.com/Oneflow-Inc/oneflow/pull/5310)
- [**enhancement**][**build**][**python**] Remove ONNX from setup py [5297](https://github.com/Oneflow-Inc/oneflow/pull/5297)
- [**enhancement**][**python**][**interface**] [add] zeropad2d [5278](https://github.com/Oneflow-Inc/oneflow/pull/5278)
- [**feature**][**system**][**python**][**interface**] Lazy nn.Graph FeedInputOpExpr [5458](https://github.com/Oneflow-Inc/oneflow/pull/5458)
- [**feature**][**python**][**interface**] integrate nn.image.flip [5411](https://github.com/Oneflow-Inc/oneflow/pull/5411)
- [**bug**][**python**] Fix issues in point of MultiClientSession [5469](https://github.com/Oneflow-Inc/oneflow/pull/5469)
- [**enhancement**][**bug**][**python**] update HasAllMultiClientEnvVars() [5459](https://github.com/Oneflow-Inc/oneflow/pull/5459)
- [**enhancement**][**python**] Add in_top_k function [5428](https://github.com/Oneflow-Inc/oneflow/pull/5428)
- [**enhancement**][**python**] Dev add docstring [5449](https://github.com/Oneflow-Inc/oneflow/pull/5449)
- [**feature**][**api**][**python**] MultiClientSession [5407](https://github.com/Oneflow-Inc/oneflow/pull/5407)
- [**documentation**][**python**] remove --user [5431](https://github.com/Oneflow-Inc/oneflow/pull/5431)
- [**feature**][**python**][**interface**] nn.Graph python [5309](https://github.com/Oneflow-Inc/oneflow/pull/5309)
- [**feature**][**python**][**interface**] Fea/nn graph/graph name [5413](https://github.com/Oneflow-Inc/oneflow/pull/5413)
- [**bug**][**python**][**interface**] rm nn.Graph.train [5424](https://github.com/Oneflow-Inc/oneflow/pull/5424)
- [**op**][**documentation**][**api**][**python**][**interface**] add bernoulli module [5353](https://github.com/Oneflow-Inc/oneflow/pull/5353)
- [**enhancement**][**python**] flow.S/B/P [5306](https://github.com/Oneflow-Inc/oneflow/pull/5306)
- [**enhancement**][**documentation**][**python**] Add instruction on upgrade pip [5400](https://github.com/Oneflow-Inc/oneflow/pull/5400)
- [**enhancement**][**python**] Rm oneflow export and experimental [5589](https://github.com/Oneflow-Inc/oneflow/pull/5589)
- [**bug**][**python**] Fix nn.graph.utils module conflict [5598](https://github.com/Oneflow-Inc/oneflow/pull/5598)
- [**feature**][**ci**][**python**] Update autotest framework [5520](https://github.com/Oneflow-Inc/oneflow/pull/5520)
- [**enhancement**][**python**] copy of_proto_python_dir to compatible_single_client_python [5539](https://github.com/Oneflow-Inc/oneflow/pull/5539)
- [**enhancement**][**api**][**python**] del default env init [5537](https://github.com/Oneflow-Inc/oneflow/pull/5537)
- [**enhancement**][**python**] Fix single client using same glog file [5535](https://github.com/Oneflow-Inc/oneflow/pull/5535)
- [**bug**][**api**][**python**] Fix Session TryClose [5531](https://github.com/Oneflow-Inc/oneflow/pull/5531)
- [**enhancement**][**feature**][**python**] split vector-matrix norm [5478](https://github.com/Oneflow-Inc/oneflow/pull/5478)
- [**feature**][**eager**][**op**][**python**][**interface**] Add more upsample kernel [5382](https://github.com/Oneflow-Inc/oneflow/pull/5382)
- [**enhancement**][**feature**][**test**][**python**] add torchstyle unittest [5489](https://github.com/Oneflow-Inc/oneflow/pull/5489)
- [**feature**][**system**][**python**] nn.Graph with training [5662](https://github.com/Oneflow-Inc/oneflow/pull/5662)
- [**enhancement**][**feature**][**python**] Fea/nn graph/block proxy func [5727](https://github.com/Oneflow-Inc/oneflow/pull/5727)
- [**enhancement**][**api**][**python**] consistent_tensor_to_api [5703](https://github.com/Oneflow-Inc/oneflow/pull/5703)
- [**feature**][**eager**][**op**][**python**] Dev Align torch avgpool [5610](https://github.com/Oneflow-Inc/oneflow/pull/5610)
- [**enhancement**][**python**] fix circular deps of sbp python module [5706](https://github.com/Oneflow-Inc/oneflow/pull/5706)
- [**documentation**][**python**] [part5]Remove singleclient outdated api [5674](https://github.com/Oneflow-Inc/oneflow/pull/5674)
- [**enhancement**][**python**] [part4]Remove singleclient outdated api [5672](https://github.com/Oneflow-Inc/oneflow/pull/5672)
- [**bug**][**op**][**python**] remove outdated code in conv3d [5696](https://github.com/Oneflow-Inc/oneflow/pull/5696)
- [**enhancement**][**test**][**python**] enlarge tolerance of dataloader test [5689](https://github.com/Oneflow-Inc/oneflow/pull/5689)
- [**enhancement**][**test**][**python**] add autotest for some math ops [5646](https://github.com/Oneflow-Inc/oneflow/pull/5646)
- [**feature**][**python**] nn.Graph optimizer part 2: add L2, pass job complete, refactor [5604](https://github.com/Oneflow-Inc/oneflow/pull/5604)
- [**enhancement**][**python**] Add clip_grad_norm [5299](https://github.com/Oneflow-Inc/oneflow/pull/5299)
- [**purge**][**python**] Remove Single-Client API in oneflow default python [5827](https://github.com/Oneflow-Inc/oneflow/pull/5827)
- [**bug**][**python**] Fix ddp grad size [5834](https://github.com/Oneflow-Inc/oneflow/pull/5834)
- [**enhancement**][**feature**][**python**] Dev RMSprop graph conf [5768](https://github.com/Oneflow-Inc/oneflow/pull/5768)
- [**enhancement**][**purge**][**eager**][**python**] remove scale arg in optimizer [5821](https://github.com/Oneflow-Inc/oneflow/pull/5821)
- [**enhancement**][**feature**][**python**] graph/block io check [5803](https://github.com/Oneflow-Inc/oneflow/pull/5803)
- [**enhancement**][**feature**][**python**] Dev adam graph conf [5709](https://github.com/Oneflow-Inc/oneflow/pull/5709)
- [**purge**][**python**] [part10]Remove singleclient outdated api [5756](https://github.com/Oneflow-Inc/oneflow/pull/5756)
- [**feature**][**api**][**python**] better repr of nn.Graph for debug [5762](https://github.com/Oneflow-Inc/oneflow/pull/5762)
- [**bug**][**python**] fix weight decay in RMSprop [5755](https://github.com/Oneflow-Inc/oneflow/pull/5755)
- [**purge**][**python**] [part9]Remove singleclient outdated api [5752](https://github.com/Oneflow-Inc/oneflow/pull/5752)
- [**purge**][**python**] [part8]Remove singleclient outdated api [5750](https://github.com/Oneflow-Inc/oneflow/pull/5750)
- [**documentation**][**python**] add first batch of methods in oneflow.nn.functional namespace [5693](https://github.com/Oneflow-Inc/oneflow/pull/5693)
- [**purge**][**python**] [part6]Remove singleclient outdated api [5704](https://github.com/Oneflow-Inc/oneflow/pull/5704)
- [**bug**][**python**] use default_generator.seed() as random_seed in init [5721](https://github.com/Oneflow-Inc/oneflow/pull/5721)
- [**bug**][**system**][**python**] ddp broadcast params and buffers [5913](https://github.com/Oneflow-Inc/oneflow/pull/5913)
- [**enhancement**][**test**][**python**] Add consistent tensor requires grad test [5925](https://github.com/Oneflow-Inc/oneflow/pull/5925)
- [**bug**][**python**] wrap flow.nn.init.\* with flow.no_grad() [5932](https://github.com/Oneflow-Inc/oneflow/pull/5932)
- [**feature**][**api**][**python**][**interface**] add clip_grad to optimizer [5817](https://github.com/Oneflow-Inc/oneflow/pull/5817)
- [**enhancement**][**ci**][**op**][**test**][**python**] add randperm with test and docs [5680](https://github.com/Oneflow-Inc/oneflow/pull/5680)
- [**feature**][**api**][**python**] Fea/nn graph/ lr_schedule(and cosine lr_sch) and opt_group [5846](https://github.com/Oneflow-Inc/oneflow/pull/5846)
- [**bug**][**python**] fix bug of SyncOnMasterFn atexit [5909](https://github.com/Oneflow-Inc/oneflow/pull/5909)
- [**purge**][**python**] Delete single client nn modules [6061](https://github.com/Oneflow-Inc/oneflow/pull/6061)
- [**enhancement**][**python**] Move framework.distribute to env [6022](https://github.com/Oneflow-Inc/oneflow/pull/6022)
- [**bug**][**python**] skip sync when abnormally exiting [6025](https://github.com/Oneflow-Inc/oneflow/pull/6025)
- [**feature**][**python**] Fea/nn graph/warmup amp config [5969](https://github.com/Oneflow-Inc/oneflow/pull/5969)
- [**documentation**][**python**] add optimizer api docs [6131](https://github.com/Oneflow-Inc/oneflow/pull/6131)
- [**documentation**][**python**] add_tensor_api_doc [6127](https://github.com/Oneflow-Inc/oneflow/pull/6127)
- [**bug**][**python**] Fix test_grid_sample.py and test_affine_grid.py threshold [6125](https://github.com/Oneflow-Inc/oneflow/pull/6125)
- [**documentation**][**api**][**python**] add doc of graph [6093](https://github.com/Oneflow-Inc/oneflow/pull/6093)
- [**bug**][**python**] Fix make of_format fail in ubuntu [6120](https://github.com/Oneflow-Inc/oneflow/pull/6120)
- [**feature**][**api**][**python**][**interface**] Fea/graph helpers [6088](https://github.com/Oneflow-Inc/oneflow/pull/6088)
- [**enhancement**][**eager**][**python**][**interface**] Use flow.randint in dataloader [6086](https://github.com/Oneflow-Inc/oneflow/pull/6086)
- [**feature**][**eager**][**api**][**python**][**interface**] Import oneflow as torch [6076](https://github.com/Oneflow-Inc/oneflow/pull/6076)
- [**enhancement**][**test**][**api**][**python**][**refactor**] rename OfrecordReader to OFRcordReader [6090](https://github.com/Oneflow-Inc/oneflow/pull/6090)
- [**purge**][**python**][**need-single-client-tests**] Delete single client nn modules [6082](https://github.com/Oneflow-Inc/oneflow/pull/6082)
- [**enhancement**][**python**] flow.load tolerates FileNotFound fault [6083](https://github.com/Oneflow-Inc/oneflow/pull/6083)
- [**feature**][**python**] Fea/pipeline in graph [6105](https://github.com/Oneflow-Inc/oneflow/pull/6105)
- [**enhancement**][**test**][**python**] graph activation checkpointing [6192](https://github.com/Oneflow-Inc/oneflow/pull/6192)
- [**enhancement**][**feature**][**op**][**python**] rnn test [6165](https://github.com/Oneflow-Inc/oneflow/pull/6165)

New in Ops:

- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part2: Add partial unary and math functional apis [5218](https://github.com/Oneflow-Inc/oneflow/pull/5218)
- [**enhancement**][**bug**][**op**][**interface**] Refine deconv kernel [5229](https://github.com/Oneflow-Inc/oneflow/pull/5229)
- [**enhancement**][**op**][**api**][**interface**] add ReflectionPad2d [5172](https://github.com/Oneflow-Inc/oneflow/pull/5172)
- [**feature**][**eager**][**op**][**api**][**interface**] crossentropyloss and nllloss support ignore_index [5195](https://github.com/Oneflow-Inc/oneflow/pull/5195)
- [**feature**][**eager**][**op**][**api**][**interface**] Yejiaojiao/dev bcewithlogitsloss [5173](https://github.com/Oneflow-Inc/oneflow/pull/5173)
- [**bug**][**ci**][**op**] Dev user op set default is_dynamic [5223](https://github.com/Oneflow-Inc/oneflow/pull/5223)
- [**enhancement**][**op**] add magic method for pow [5199](https://github.com/Oneflow-Inc/oneflow/pull/5199)
- [**enhancement**][**op**][**interface**] add cpu version of upsampling [5194](https://github.com/Oneflow-Inc/oneflow/pull/5194)
- [**enhancement**][**bug**][**op**][**api**][**interface**] add ReplicationPad2d [5148](https://github.com/Oneflow-Inc/oneflow/pull/5148)
- [**feature**][**eager**][**op**][**api**][**interface**] add kldivloss module [5155](https://github.com/Oneflow-Inc/oneflow/pull/5155)
- [**feature**][**eager**][**op**][**documentation**][**build**][**api**][**interface**] Add floor module and the corresponding testcases [4964](https://github.com/Oneflow-Inc/oneflow/pull/4964)
- [**enhancement**][**feature**][**op**] Dev conv1d module [5280](https://github.com/Oneflow-Inc/oneflow/pull/5280)
- [**enhancement**][**op**] Add ctc_greedy_decoder op [5294](https://github.com/Oneflow-Inc/oneflow/pull/5294)
- [**enhancement**][**op**][**system**] Dev remove default grad func [5320](https://github.com/Oneflow-Inc/oneflow/pull/5320)
- [**enhancement**][**op**][**system**] Add pad grad func. [5354](https://github.com/Oneflow-Inc/oneflow/pull/5354)
- [**enhancement**][**op**][**system**] Add gradient funcs. [5348](https://github.com/Oneflow-Inc/oneflow/pull/5348)
- [**feature**][**purge**][**bug**][**eager**][**op**][**interface**] fix upsample nearest bug [5347](https://github.com/Oneflow-Inc/oneflow/pull/5347)
- [**enhancement**][**op**][**system**] [Functional] Part7: Migrate pooling ops [5253](https://github.com/Oneflow-Inc/oneflow/pull/5253)
- [**enhancement**][**op**] nvjpeg hardware acc [5240](https://github.com/Oneflow-Inc/oneflow/pull/5240)
- [**enhancement**][**feature**][**ci**][**eager**][**op**][**api**][**interface**] Add bmm module [5334](https://github.com/Oneflow-Inc/oneflow/pull/5334)
- [**enhancement**][**eager**][**op**] Dev image decode eager [5333](https://github.com/Oneflow-Inc/oneflow/pull/5333)
- [**enhancement**][**op**] Optimize softmax warp impl [4977](https://github.com/Oneflow-Inc/oneflow/pull/4977)
- [**enhancement**][**eager**][**op**] Dev tensor buffer eager [5317](https://github.com/Oneflow-Inc/oneflow/pull/5317)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part6: Migrate conv op [5252](https://github.com/Oneflow-Inc/oneflow/pull/5252)
- [**enhancement**][**eager**][**op**] Dev sort eager [5284](https://github.com/Oneflow-Inc/oneflow/pull/5284)
- [**enhancement**][**bug**][**op**][**api**] fix bceloss bug in default weight and reduction [5303](https://github.com/Oneflow-Inc/oneflow/pull/5303)
- [**bug**][**eager**][**op**] remove redundant assert and check [5264](https://github.com/Oneflow-Inc/oneflow/pull/5264)
- [**enhancement**][**bug**][**ci**][**op**] fix bceloss bug about weight [5269](https://github.com/Oneflow-Inc/oneflow/pull/5269)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part5: Migrate nn ops [5249](https://github.com/Oneflow-Inc/oneflow/pull/5249)
- [**enhancement**][**eager**][**op**] Dev argsort eager [5273](https://github.com/Oneflow-Inc/oneflow/pull/5273)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part4: Migrate array ops [5247](https://github.com/Oneflow-Inc/oneflow/pull/5247)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part3: Migrate binary and activation ops [5246](https://github.com/Oneflow-Inc/oneflow/pull/5246)
- [**bug**][**ci**][**op**][**test**] Dev fix rmsprop ci fail [5481](https://github.com/Oneflow-Inc/oneflow/pull/5481)
- [**enhancement**][**op**] add inplace method: Tensor.sin\_ [5471](https://github.com/Oneflow-Inc/oneflow/pull/5471)
- [**bug**][**op**] hotfix image_batch_align [5461](https://github.com/Oneflow-Inc/oneflow/pull/5461)
- [**enhancement**][**eager**][**op**][**interface**] Dev maxpool series op 123d [5244](https://github.com/Oneflow-Inc/oneflow/pull/5244)
- [**bug**][**op**] fix pool gpu kernel [5446](https://github.com/Oneflow-Inc/oneflow/pull/5446)
- [**feature**][**eager**][**op**][**api**][**interface**] add pixelshufflev2 module [5383](https://github.com/Oneflow-Inc/oneflow/pull/5383)
- [**enhancement**][**feature**][**ci**][**eager**][**op**][**documentation**][**api**][**interface**] Add flow xxx and tensor xxx autotest [5386](https://github.com/Oneflow-Inc/oneflow/pull/5386)
- [**enhancement**][**feature**][**eager**][**op**][**api**][**interface**] Modules chunk [5324](https://github.com/Oneflow-Inc/oneflow/pull/5324)
- [**enhancement**][**eager**][**op**] add image normalize for eager [5402](https://github.com/Oneflow-Inc/oneflow/pull/5402)
- [**enhancement**][**eager**][**op**] Dev batch align module [5401](https://github.com/Oneflow-Inc/oneflow/pull/5401)
- [**enhancement**][**eager**][**op**] add coco reader module [5391](https://github.com/Oneflow-Inc/oneflow/pull/5391)
- [**enhancement**][**wip**][**op**] Restruct Elementwise kernel [4130](https://github.com/Oneflow-Inc/oneflow/pull/4130)
- [**bug**][**op**] Fix DecodeRandom reuse mem [5606](https://github.com/Oneflow-Inc/oneflow/pull/5606)
- [**enhancement**][**op**] Align pytorch maxpool [5525](https://github.com/Oneflow-Inc/oneflow/pull/5525)
- [**enhancement**][**bottleneck**][**eager**][**op**][**api**] implementation of constantpad-3d op [5529](https://github.com/Oneflow-Inc/oneflow/pull/5529)
- [**enhancement**][**eager**][**op**] Add scale size for resize [5509](https://github.com/Oneflow-Inc/oneflow/pull/5509)
- [**enhancement**][**op**][**api**][**refactor**] Dev optimize tensor setitem [5501](https://github.com/Oneflow-Inc/oneflow/pull/5501)
- [**enhancement**][**op**] register uint8 dtypeto support dataloader [5499](https://github.com/Oneflow-Inc/oneflow/pull/5499)
- [**enhancement**][**op**] Add unique.cuh [5487](https://github.com/Oneflow-Inc/oneflow/pull/5487)
- [**enhancement**][**op**][**api**][**interface**] Dev ofrecord auto truncating [5412](https://github.com/Oneflow-Inc/oneflow/pull/5412)
- [**feature**][**op**][**system**][**interface**] Feat: LazyInterpret::ApplyImpl support SourceUserOpExpr and Copy [5711](https://github.com/Oneflow-Inc/oneflow/pull/5711)
- [**enhancement**][**op**][**interface**] Dev logical_and/or modules [5636](https://github.com/Oneflow-Inc/oneflow/pull/5636)
- [**enhancement**][**op**] support any number positional arguments for ones and zeros op [5698](https://github.com/Oneflow-Inc/oneflow/pull/5698)
- [**enhancement**][**feature**][**eager**][**op**] Add conv3d Module [5327](https://github.com/Oneflow-Inc/oneflow/pull/5327)
- [**feature**][**eager**][**op**][**api**][**interface**] add batchnorm3d module [5631](https://github.com/Oneflow-Inc/oneflow/pull/5631)
- [**bug**][**eager**][**op**] fix reduce min max backward bug [5651](https://github.com/Oneflow-Inc/oneflow/pull/5651)
- [**enhancement**][**op**] Debug dim scatter [5371](https://github.com/Oneflow-Inc/oneflow/pull/5371)
- [**enhancement**][**op**][**interface**] Dev eye [5583](https://github.com/Oneflow-Inc/oneflow/pull/5583)
- [**enhancement**][**eager**][**op**] Dev minimum maximum [5576](https://github.com/Oneflow-Inc/oneflow/pull/5576)
- [**enhancement**][**op**] Restruct activation grad op [5669](https://github.com/Oneflow-Inc/oneflow/pull/5669)
- [**enhancement**][**feature**][**eager**][**op**] Rewrite activation function [5465](https://github.com/Oneflow-Inc/oneflow/pull/5465)
- [**bug**][**op**][**documentation**] add oneflow.cat for documentation [5621](https://github.com/Oneflow-Inc/oneflow/pull/5621)
- [**enhancement**][**op**] Lcy logsoftmax [5746](https://github.com/Oneflow-Inc/oneflow/pull/5746)
- [**feature**][**op**][**need-simple-ci**] Feat empty op [5659](https://github.com/Oneflow-Inc/oneflow/pull/5659)
- [**enhancement**][**eager**][**op**] Dev split [5714](https://github.com/Oneflow-Inc/oneflow/pull/5714)
- [**enhancement**][**op**][**interface**] add index_select op [5661](https://github.com/Oneflow-Inc/oneflow/pull/5661)
- [**bug**][**op**] fix nvjpeg hw acc [5851](https://github.com/Oneflow-Inc/oneflow/pull/5851)
- [**enhancement**][**op**] Remove move in conv_cudnn [5828](https://github.com/Oneflow-Inc/oneflow/pull/5828)
- [**enhancement**][**op**][**interface**] Dev logical_xor module [5694](https://github.com/Oneflow-Inc/oneflow/pull/5694)
- [**bug**][**eager**][**op**] fix squeeze [5808](https://github.com/Oneflow-Inc/oneflow/pull/5808)
- [**enhancement**][**op**] Get parallel_id and parallel_num through rank and world size in DDP [5717](https://github.com/Oneflow-Inc/oneflow/pull/5717)
- [**bug**][**eager**][**op**] delete interpolate int type [5805](https://github.com/Oneflow-Inc/oneflow/pull/5805)
- [**bug**][**op**] Fix bug in scatter [5743](https://github.com/Oneflow-Inc/oneflow/pull/5743)
- [**enhancement**][**op**] Refactor: remove module not required, call function directly [5754](https://github.com/Oneflow-Inc/oneflow/pull/5754)
- [**enhancement**][**op**] Remove modules not required(tan, erfc, log1p, scatter_nd) [5791](https://github.com/Oneflow-Inc/oneflow/pull/5791)
- [**enhancement**][**op**] Refactor scatter, clamp and pow in cpp instead of in python [5715](https://github.com/Oneflow-Inc/oneflow/pull/5715)
- [**enhancement**][**op**] Rm useless code in gather files [5687](https://github.com/Oneflow-Inc/oneflow/pull/5687)
- [**enhancement**][**eager**][**op**] change flip_code to scalar [5786](https://github.com/Oneflow-Inc/oneflow/pull/5786)
- [**enhancement**][**bug**][**op**][**interface**] fix upsample bug [5753](https://github.com/Oneflow-Inc/oneflow/pull/5753)
- [**bug**][**op**][**interface**] Quick fix Lazy nn.Graph input/output OpConf.BlobConf.is_dynamic [5767](https://github.com/Oneflow-Inc/oneflow/pull/5767)
- [**enhancement**][**bug**][**eager**][**op**] fix argwhere 0-dim bug [5760](https://github.com/Oneflow-Inc/oneflow/pull/5760)
- [**enhancement**][**eager**][**op**] delete unused code [5744](https://github.com/Oneflow-Inc/oneflow/pull/5744)
- [**feature**][**op**] Export fused_scale_tril op [5933](https://github.com/Oneflow-Inc/oneflow/pull/5933)
- [**bug**][**op**] Fix backward bug in 3d [5908](https://github.com/Oneflow-Inc/oneflow/pull/5908)
- [**bug**][**op**] Fix one_hot api limit [5927](https://github.com/Oneflow-Inc/oneflow/pull/5927)
- [**enhancement**][**eager**][**op**] Dev where scalar [5797](https://github.com/Oneflow-Inc/oneflow/pull/5797)
- [**bug**][**op**] fix grad error [5914](https://github.com/Oneflow-Inc/oneflow/pull/5914)
- [**feature**][**bug**][**op**] Fix inplace op circle reference bug [5910](https://github.com/Oneflow-Inc/oneflow/pull/5910)
- [**enhancement**][**op**] Move the judgment content to c++， And add scalar fmod [5854](https://github.com/Oneflow-Inc/oneflow/pull/5854)
- [**enhancement**][**op**] Support combined_margin_loss op in flow.nn.modules [5830](https://github.com/Oneflow-Inc/oneflow/pull/5830)
- [**enhancement**][**op**][**api**][**interface**] functional_one_hot [5315](https://github.com/Oneflow-Inc/oneflow/pull/5315)
- [**enhancement**][**op**] Dev scalar op [5778](https://github.com/Oneflow-Inc/oneflow/pull/5778)
- [**bug**][**eager**][**op**] fix gather kernel 0 shape [5888](https://github.com/Oneflow-Inc/oneflow/pull/5888)
- [**enhancement**][**op**] add l2_normalize for mutl-client interfaces [5859](https://github.com/Oneflow-Inc/oneflow/pull/5859)
- [**feature**][**op**] Export function softmax_cross_entropy [6056](https://github.com/Oneflow-Inc/oneflow/pull/6056)
- [**enhancement**][**op**] Add int attr for functional adaptive average pool [6059](https://github.com/Oneflow-Inc/oneflow/pull/6059)
- [**enhancement**][**op**][**interface**] dev full op [5955](https://github.com/Oneflow-Inc/oneflow/pull/5955)
- [**bug**][**eager**][**op**] fix 0dim inplace add [6029](https://github.com/Oneflow-Inc/oneflow/pull/6029)
- [**feature**][**op**][**system**][**interface**] Feat: nn.Graph image gpu decoder [6014](https://github.com/Oneflow-Inc/oneflow/pull/6014)
- [**enhancement**][**op**][**interface**] dev optim_optim_lr_scheduler_multisteplr [5975](https://github.com/Oneflow-Inc/oneflow/pull/5975)
- [**enhancement**][**op**] NopKernel [6035](https://github.com/Oneflow-Inc/oneflow/pull/6035)
- [**enhancement**][**eager**][**op**][**api**] Dev tril op [6005](https://github.com/Oneflow-Inc/oneflow/pull/6005)
- [**enhancement**][**op**] dev unfold and fold [5675](https://github.com/Oneflow-Inc/oneflow/pull/5675)
- [**enhancement**][**op**] ResNet CUDA Graphs [6018](https://github.com/Oneflow-Inc/oneflow/pull/6018)
- [**enhancement**][**feature**][**op**] add broadcast pow [6013](https://github.com/Oneflow-Inc/oneflow/pull/6013)
- [**enhancement**][**op**][**interface**] init of op diag [5298](https://github.com/Oneflow-Inc/oneflow/pull/5298)
- [**op**][**documentation**][**api**] Fix api document bug [6009](https://github.com/Oneflow-Inc/oneflow/pull/6009)
- [**enhancement**][**op**] Dev fused functional [5954](https://github.com/Oneflow-Inc/oneflow/pull/5954)
- [**bug**][**op**][**build**] Add nvcc flag -Werror cross-execution-space-call [6002](https://github.com/Oneflow-Inc/oneflow/pull/6002)
- [**bug**][**op**] Fix Normalization grad function [5993](https://github.com/Oneflow-Inc/oneflow/pull/5993)
- [**enhancement**][**feature**][**eager**][**op**][**test**][**interface**] Add fused self attention [5966](https://github.com/Oneflow-Inc/oneflow/pull/5966)
- [**enhancement**][**bug**][**ci**][**eager**][**op**][**api**][**interface**] Try to fix var bug [5973](https://github.com/Oneflow-Inc/oneflow/pull/5973)
- [**enhancement**][**feature**][**eager**][**op**][**interface**] add prod op [5867](https://github.com/Oneflow-Inc/oneflow/pull/5867)
- [**enhancement**][**eager**][**op**][**api**] add glu op [6065](https://github.com/Oneflow-Inc/oneflow/pull/6065)
- [**enhancement**][**op**] Align Torch.nn.functional poolXd [6184](https://github.com/Oneflow-Inc/oneflow/pull/6184)
- [**bug**][**eager**][**op**] fix backward index for gamma beta [6149](https://github.com/Oneflow-Inc/oneflow/pull/6149)
- [**bug**][**op**][**system**] Fix BroadcastMatmulGrad bug [6168](https://github.com/Oneflow-Inc/oneflow/pull/6168)
- [**enhancement**][**op**][**api**] Add Int support for functional.avg/maxpool [6174](https://github.com/Oneflow-Inc/oneflow/pull/6174)
- [**bug**][**eager**][**op**][**api**][**interface**] align dropout api name with pytorch [6170](https://github.com/Oneflow-Inc/oneflow/pull/6170)
- [**enhancement**][**op**] support inplace operation for hardsigmoid [6137](https://github.com/Oneflow-Inc/oneflow/pull/6137)
- [**enhancement**][**bug**][**op**] Fix do bias correction in Adam/AdamW [5960](https://github.com/Oneflow-Inc/oneflow/pull/5960)
- [**bug**][**eager**][**op**][**api**][**interface**] fix repeat 0-dim tensor bug [6150](https://github.com/Oneflow-Inc/oneflow/pull/6150)
- [**enhancement**][**bug**][**op**] Fix select_first_grad bug [6142](https://github.com/Oneflow-Inc/oneflow/pull/6142)
- [**bug**][**ci**][**eager**][**op**][**documentation**][**interface**] Add clipgrad doc and contiguous [6130](https://github.com/Oneflow-Inc/oneflow/pull/6130)
- [**bug**][**op**] Fix eager optim dynamic attr bug [6111](https://github.com/Oneflow-Inc/oneflow/pull/6111)
- [**enhancement**][**op**] Support grid_sample and affine_grid operator [6038](https://github.com/Oneflow-Inc/oneflow/pull/6038)
- [**op**][**documentation**] Export apis for documentation [6068](https://github.com/Oneflow-Inc/oneflow/pull/6068)
- [**enhancement**][**feature**][**bug**][**ci**][**eager**][**op**][**documentation**][**interface**] transfer python function to c++ method [6114](https://github.com/Oneflow-Inc/oneflow/pull/6114)
- [**op**][**documentation**] Dev functional batch_gather [6233](https://github.com/Oneflow-Inc/oneflow/pull/6233)
- [**enhancement**][**op**][**test**] fix cross_entropy_loss and its test [5799](https://github.com/Oneflow-Inc/oneflow/pull/5799)
- [**bug**][**op**] Use attr nd_sbp to check consistent [6222](https://github.com/Oneflow-Inc/oneflow/pull/6222)
- [**enhancement**][**op**] Dev fused bn functional [6077](https://github.com/Oneflow-Inc/oneflow/pull/6077)
- [**enhancement**][**op**] support default value in intlist [6201](https://github.com/Oneflow-Inc/oneflow/pull/6201)
- [**bug**][**op**] fix sparse_softmax get_nd_sbp [6203](https://github.com/Oneflow-Inc/oneflow/pull/6203)
- [**bug**][**op**] Fix bug in model fused update [6197](https://github.com/Oneflow-Inc/oneflow/pull/6197)
- [**enhancement**][**op**][**system**][**refactor**] Optimize tensor getitem. [5433](https://github.com/Oneflow-Inc/oneflow/pull/5433)

New in Eager:

- [**enhancement**][**eager**][**interface**] Reconstruct module files [5251](https://github.com/Oneflow-Inc/oneflow/pull/5251)
- [**bug**][**eager**][**documentation**][**interface**] Fix conv module bug [5245](https://github.com/Oneflow-Inc/oneflow/pull/5245)
- [**bug**][**ci**][**eager**][**interface**] Fix bce withlogitloss ci error [5237](https://github.com/Oneflow-Inc/oneflow/pull/5237)
- [**feature**][**eager**][**api**][**interface**] module BCELoss [5144](https://github.com/Oneflow-Inc/oneflow/pull/5144)
- [**enhancement**][**feature**][**eager**][**api**][**interface**] Dev norm op [5178](https://github.com/Oneflow-Inc/oneflow/pull/5178)
- [**enhancement**][**bug**][**eager**] Fix stack module [5222](https://github.com/Oneflow-Inc/oneflow/pull/5222)
- [**enhancement**][**feature**][**eager**] Support different dtype of equal module [5214](https://github.com/Oneflow-Inc/oneflow/pull/5214)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**][**interface**] Add nllloss backward [5210](https://github.com/Oneflow-Inc/oneflow/pull/5210)
- [**enhancement**][**eager**][**api**][**upload-core**] Decouple FileSystem and IOConf [5162](https://github.com/Oneflow-Inc/oneflow/pull/5162)
- [**enhancement**][**ci**][**eager**] Set lower precision avoid ci failing [5200](https://github.com/Oneflow-Inc/oneflow/pull/5200)
- [**eager**][**documentation**] Add hint when apply FunctionNode second time [5369](https://github.com/Oneflow-Inc/oneflow/pull/5369)
- [**enhancement**][**feature**][**bug**][**ci**][**eager**][**documentation**][**api**] Fix upsample bilinear bug [5366](https://github.com/Oneflow-Inc/oneflow/pull/5366)
- [**bug**][**eager**] Fix not contiguous ndarray to tensor bug [5351](https://github.com/Oneflow-Inc/oneflow/pull/5351)
- [**enhancement**][**eager**][**system**] Infer consistent tensor meta [5118](https://github.com/Oneflow-Inc/oneflow/pull/5118)
- [**feature**][**eager**] Feat graph autograd engine [5296](https://github.com/Oneflow-Inc/oneflow/pull/5296)
- [**enhancement**][**eager**][**interface**] Dev type as module [5349](https://github.com/Oneflow-Inc/oneflow/pull/5349)
- [**feature**][**eager**][**documentation**][**api**][**interface**] Add new ones module [5342](https://github.com/Oneflow-Inc/oneflow/pull/5342)
- [**enhancement**][**bug**][**eager**] Fix logical slice assign dtype [5339](https://github.com/Oneflow-Inc/oneflow/pull/5339)
- [**bug**][**ci**][**eager**][**documentation**][**api**][**interface**] Fix where module bug [5300](https://github.com/Oneflow-Inc/oneflow/pull/5300)
- [**bug**][**ci**][**eager**][**documentation**][**api**] Fix l1loss ci error [5307](https://github.com/Oneflow-Inc/oneflow/pull/5307)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**][**interface**] Qi's First Edit of deleting "print" and ".numpy" [5129](https://github.com/Oneflow-Inc/oneflow/pull/5129)
- [**feature**][**eager**][**refactor**] Separate autograd meta to tensor [5267](https://github.com/Oneflow-Inc/oneflow/pull/5267)
- [**feature**][**eager**][**api**][**interface**] add tile module [5234](https://github.com/Oneflow-Inc/oneflow/pull/5234)
- [**enhancement**][**eager**] Release lambda function to reuse tensor memory [5266](https://github.com/Oneflow-Inc/oneflow/pull/5266)
- [**feature**][**bug**][**eager**][**documentation**] Fix default value not set bug [5483](https://github.com/Oneflow-Inc/oneflow/pull/5483)
- [**enhancement**][**eager**][**interface**] [Add] gather_nd scatter_nd [5422](https://github.com/Oneflow-Inc/oneflow/pull/5422)
- [**enhancement**][**bug**][**eager**] fix param [5473](https://github.com/Oneflow-Inc/oneflow/pull/5473)
- [**bug**][**eager**] Fix Tensor.grad setter bug [5462](https://github.com/Oneflow-Inc/oneflow/pull/5462)
- [**enhancement**][**eager**] Rename now_grad_arg to current_grad [5466](https://github.com/Oneflow-Inc/oneflow/pull/5466)
- [**eager**][**test**][**documentation**][**interface**] Add autotest part1 [5436](https://github.com/Oneflow-Inc/oneflow/pull/5436)
- [**enhancement**][**eager**] Use functional copy instead of op_builder [5460](https://github.com/Oneflow-Inc/oneflow/pull/5460)
- [**bottleneck**][**bug**][**eager**][**interface**] fix -1 index not support bug [5448](https://github.com/Oneflow-Inc/oneflow/pull/5448)
- [**bug**][**ci**][**eager**][**documentation**][**api**] Fix concat backward bug [5443](https://github.com/Oneflow-Inc/oneflow/pull/5443)
- [**enhancement**][**bug**][**ci**][**eager**] Add autograd engine warning [5444](https://github.com/Oneflow-Inc/oneflow/pull/5444)
- [**feature**][**eager**][**api**][**interface**] Smoothl1loss [5256](https://github.com/Oneflow-Inc/oneflow/pull/5256)
- [**enhancement**][**bottleneck**][**eager**] remove device dtype params [5434](https://github.com/Oneflow-Inc/oneflow/pull/5434)
- [**bug**][**ci**][**eager**][**documentation**][**interface**] Delete maxpool failed test [5409](https://github.com/Oneflow-Inc/oneflow/pull/5409)
- [**enhancement**][**eager**][**api**] Add tensor grad assginment [5379](https://github.com/Oneflow-Inc/oneflow/pull/5379)
- [**enhancement**][**bug**][**eager**] fix-abs [5398](https://github.com/Oneflow-Inc/oneflow/pull/5398)
- [**enhancement**][**bug**][**eager**][**interface**] Fix bn track running stats [5393](https://github.com/Oneflow-Inc/oneflow/pull/5393)
- [**enhancement**][**bug**][**eager**] Support uint dtype of constant op [5396](https://github.com/Oneflow-Inc/oneflow/pull/5396)
- [**enhancement**][**bug**][**eager**][**documentation**][**interface**] Delete useless code upsample [5392](https://github.com/Oneflow-Inc/oneflow/pull/5392)
- [**enhancement**][**ci**][**eager**][**interface**] add flow.view [5301](https://github.com/Oneflow-Inc/oneflow/pull/5301)
- [**enhancement**][**bug**][**ci**][**eager**][**api**][**interface**] Add masked select module [5356](https://github.com/Oneflow-Inc/oneflow/pull/5356)
- [**bug**][**eager**][**interface**] Fix batchnorm backward bug [5602](https://github.com/Oneflow-Inc/oneflow/pull/5602)
- [**enhancement**][**eager**] Support weight_dacay(l2 actually) [5587](https://github.com/Oneflow-Inc/oneflow/pull/5587)
- [**feature**][**eager**][**documentation**][**api**] Add new autotest [5588](https://github.com/Oneflow-Inc/oneflow/pull/5588)
- [**enhancement**][**eager**][**documentation**][**api**] Dev fmod [5404](https://github.com/Oneflow-Inc/oneflow/pull/5404)
- [**feature**][**eager**] Support inplace add [5432](https://github.com/Oneflow-Inc/oneflow/pull/5432)
- [**feature**][**eager**][**interface**] Feat tensor stride property [5543](https://github.com/Oneflow-Inc/oneflow/pull/5543)
- [**enhancement**][**feature**][**eager**][**documentation**][**api**] Add flip module [5541](https://github.com/Oneflow-Inc/oneflow/pull/5541)
- [**feature**][**eager**] Feat module repr [5486](https://github.com/Oneflow-Inc/oneflow/pull/5486)
- [**enhancement**][**bottleneck**][**bug**][**eager**][**interface**] Fix maxpool1d params [5493](https://github.com/Oneflow-Inc/oneflow/pull/5493)
- [**enhancement**][**feature**][**eager**][**interface**] Dev flow.utils.data part1 [5406](https://github.com/Oneflow-Inc/oneflow/pull/5406)
- [**bug**][**eager**][**api**] Fix tensor getitem bug [5474](https://github.com/Oneflow-Inc/oneflow/pull/5474)
- [**enhancement**][**eager**][**need-simple-ci**] export datasets interface [5691](https://github.com/Oneflow-Inc/oneflow/pull/5691)
- [**enhancement**][**eager**][**system**] rebase [5601](https://github.com/Oneflow-Inc/oneflow/pull/5601)
- [**enhancement**][**eager**][**test**] added nn.RecordBytesDecoder with its test [5475](https://github.com/Oneflow-Inc/oneflow/pull/5475)
- [**enhancement**][**feature**][**eager**][**need-simple-ci**] 0-dim tensor support [5552](https://github.com/Oneflow-Inc/oneflow/pull/5552)
- [**enhancement**][**bug**][**eager**] rewrite slice_update backward [5677](https://github.com/Oneflow-Inc/oneflow/pull/5677)
- [**enhancement**][**bug**][**eager**][**interface**] align view input style with torch [5676](https://github.com/Oneflow-Inc/oneflow/pull/5676)
- [**enhancement**][**eager**][**interface**][**need-simple-ci**] add autotests for modules [5666](https://github.com/Oneflow-Inc/oneflow/pull/5666)
- [**enhancement**][**bottleneck**][**eager**][**interface**] Dev constantpad1d op [5579](https://github.com/Oneflow-Inc/oneflow/pull/5579)
- [**enhancement**][**eager**][**api**][**interface**] Restruct MathOps AutoTest [5654](https://github.com/Oneflow-Inc/oneflow/pull/5654)
- [**enhancement**][**bug**][**ci**][**eager**] Fix flip bug [5657](https://github.com/Oneflow-Inc/oneflow/pull/5657)
- [**bug**][**eager**][**api**][**interface**] Fix expand module bug [5650](https://github.com/Oneflow-Inc/oneflow/pull/5650)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**] Fix repeat bug [5633](https://github.com/Oneflow-Inc/oneflow/pull/5633)
- [**enhancement**][**eager**][**test**][**api**][**interface**] Add new autotest [5617](https://github.com/Oneflow-Inc/oneflow/pull/5617)
- [**enhancement**][**eager**][**api**][**interface**] Dev flow.utils.data part2 [5500](https://github.com/Oneflow-Inc/oneflow/pull/5500)
- [**enhancement**][**bug**][**eager**] make setitem device match [5835](https://github.com/Oneflow-Inc/oneflow/pull/5835)
- [**bug**][**eager**][**api**][**interface**] align reshape input param with pytorch [5804](https://github.com/Oneflow-Inc/oneflow/pull/5804)
- [**feature**][**bug**][**eager**][**api**] Align where op with torch [5850](https://github.com/Oneflow-Inc/oneflow/pull/5850)
- [**enhancement**][**bug**][**eager**][**api**] Restruct prelu op [5829](https://github.com/Oneflow-Inc/oneflow/pull/5829)
- [**bug**][**eager**][**need-simple-ci**] fix pooling ceil_mode bug [5818](https://github.com/Oneflow-Inc/oneflow/pull/5818)
- [**enhancement**][**eager**] stateful local kernel supports consistent [5789](https://github.com/Oneflow-Inc/oneflow/pull/5789)
- [**bug**][**eager**][**api**][**interface**] Fix argwhere bug [5816](https://github.com/Oneflow-Inc/oneflow/pull/5816)
- [**enhancement**][**eager**][**documentation**][**api**] dev-nonzero [5809](https://github.com/Oneflow-Inc/oneflow/pull/5809)
- [**enhancement**][**feature**][**eager**][**api**] Add fake quantize op [5690](https://github.com/Oneflow-Inc/oneflow/pull/5690)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**] Add api [5663](https://github.com/Oneflow-Inc/oneflow/pull/5663)
- [**enhancement**][**eager**] Refactor consistent infer result [5790](https://github.com/Oneflow-Inc/oneflow/pull/5790)
- [**bug**][**eager**][**need-simple-ci**] skip dataloader test [5780](https://github.com/Oneflow-Inc/oneflow/pull/5780)
- [**bug**][**eager**][**need-simple-ci**] fix 0-dim tensor.fill\_ [5771](https://github.com/Oneflow-Inc/oneflow/pull/5771)
- [**enhancement**][**eager**] Cpu mpi broadcast [5726](https://github.com/Oneflow-Inc/oneflow/pull/5726)
- [**feature**][**eager**] Feat grad mode classes [5956](https://github.com/Oneflow-Inc/oneflow/pull/5956)
- [**enhancement**][**bug**][**eager**] fix wrong names [5951](https://github.com/Oneflow-Inc/oneflow/pull/5951)
- [**enhancement**][**eager**][**system**] Local dep object pool [5953](https://github.com/Oneflow-Inc/oneflow/pull/5953)
- [**enhancement**][**eager**][**interface**] rename OpExprInterpState to AutoGradCaptureState [5918](https://github.com/Oneflow-Inc/oneflow/pull/5918)
- [**bug**][**eager**] Fix linear bug [5945](https://github.com/Oneflow-Inc/oneflow/pull/5945)
- [**bug**][**eager**] Fix tensor_meta update bug [5924](https://github.com/Oneflow-Inc/oneflow/pull/5924)
- [**enhancement**][**eager**] use flow.randperm [5928](https://github.com/Oneflow-Inc/oneflow/pull/5928)
- [**enhancement**][**eager**] consistent init/save/load [5896](https://github.com/Oneflow-Inc/oneflow/pull/5896)
- [**enhancement**][**bug**][**eager**][**documentation**][**interface**] Restruct sort and argsort op [5911](https://github.com/Oneflow-Inc/oneflow/pull/5911)
- [**enhancement**][**bug**][**eager**][**interface**] Try to fix the problem that the insightface cannot converge。 [5906](https://github.com/Oneflow-Inc/oneflow/pull/5906)
- [**enhancement**][**bug**][**eager**][**interface**] Add autotest [5899](https://github.com/Oneflow-Inc/oneflow/pull/5899)
- [**enhancement**][**eager**] The scheduler thread joins worker threads [5893](https://github.com/Oneflow-Inc/oneflow/pull/5893)
- [**enhancement**][**eager**] Bugfix async callback [5881](https://github.com/Oneflow-Inc/oneflow/pull/5881)
- [**feature**][**eager**] Feat tensor to bool [5836](https://github.com/Oneflow-Inc/oneflow/pull/5836)
- [**bug**][**eager**] Remove inplace broadcast_add [5551](https://github.com/Oneflow-Inc/oneflow/pull/5551)
- [**enhancement**][**eager**] Broadcast consistent shape and dtype [5784](https://github.com/Oneflow-Inc/oneflow/pull/5784)
- [**enhancement**][**eager**] Fix optimizer list parameters input bug [5848](https://github.com/Oneflow-Inc/oneflow/pull/5848)
- [**enhancement**][**eager**][**interface**] Dev flow.utils.data part3 [5644](https://github.com/Oneflow-Inc/oneflow/pull/5644)
- [**enhancement**][**eager**][**api**] Normalize naming of modules [6066](https://github.com/Oneflow-Inc/oneflow/pull/6066)
- [**enhancement**][**feature**][**eager**][**api**][**interface**] add trunc*normal* [6051](https://github.com/Oneflow-Inc/oneflow/pull/6051)
- [**enhancement**][**bug**][**eager**] AutoMatedTest support test module.parameter.grad [6043](https://github.com/Oneflow-Inc/oneflow/pull/6043)
- [**enhancement**][**feature**][**bug**][**eager**] add module **call** kwags [6069](https://github.com/Oneflow-Inc/oneflow/pull/6069)
- [**enhancement**][**eager**][**api**][**interface**] add tensor.item tensor.tolist [6021](https://github.com/Oneflow-Inc/oneflow/pull/6021)
- [**enhancement**][**eager**][**api**][**interface**] Export pool ops api [6047](https://github.com/Oneflow-Inc/oneflow/pull/6047)
- [**enhancement**][**bug**][**eager**][**test**][**documentation**][**interface**] Add more autotest sample [6039](https://github.com/Oneflow-Inc/oneflow/pull/6039)
- [**enhancement**][**bug**][**eager**][**system**] disable cuda_h2d stream [6020](https://github.com/Oneflow-Inc/oneflow/pull/6020)
- [**feature**][**eager**][**test**][**api**][**interface**] Add autotest codegen [6019](https://github.com/Oneflow-Inc/oneflow/pull/6019)
- [**feature**][**eager**][**documentation**] Refactor cosine lr scheduler [6000](https://github.com/Oneflow-Inc/oneflow/pull/6000)
- [**enhancement**][**eager**][**interface**] tensor.cpu/tensor.cuda [5894](https://github.com/Oneflow-Inc/oneflow/pull/5894)
- [**enhancement**][**eager**][**api**] Support consistent_tensor.to(dtype) [5991](https://github.com/Oneflow-Inc/oneflow/pull/5991)
- [**bug**][**eager**][**interface**] remove redundant codes in ModuleDict [5961](https://github.com/Oneflow-Inc/oneflow/pull/5961)
- [**bug**][**eager**] Fix LayerNorm check bug [6196](https://github.com/Oneflow-Inc/oneflow/pull/6196)
- [**enhancement**][**eager**][**api**] Change dropout api [6182](https://github.com/Oneflow-Inc/oneflow/pull/6182)
- [**enhancement**][**good for pr**][**eager**][**api**][**interface**] add: test convert dependency [6023](https://github.com/Oneflow-Inc/oneflow/pull/6023)
- [**enhancement**][**bug**][**eager**][**interface**] Fix autotest codegen bug [6171](https://github.com/Oneflow-Inc/oneflow/pull/6171)
- [**bug**][**eager**] restore instr_local_dep_object_pool_size for nccl [6160](https://github.com/Oneflow-Inc/oneflow/pull/6160)
- [**enhancement**][**eager**][**api**][**interface**] Aligin pooling op functional api names with torch [6163](https://github.com/Oneflow-Inc/oneflow/pull/6163)
- [**feature**][**bug**][**eager**][**api**][**interface**] delete file [6162](https://github.com/Oneflow-Inc/oneflow/pull/6162)
- [**bug**][**eager**] Fix optim load_state_dict bug [6152](https://github.com/Oneflow-Inc/oneflow/pull/6152)
- [**enhancement**][**eager**][**api**] add is_training to dropout functor [6148](https://github.com/Oneflow-Inc/oneflow/pull/6148)
- [**enhancement**][**eager**] Decompose nd sbp boxing [5800](https://github.com/Oneflow-Inc/oneflow/pull/5800)
- [**enhancement**][**eager**] support consistent_tensor.to(copy=True) [6122](https://github.com/Oneflow-Inc/oneflow/pull/6122)
- [**feature**][**eager**] Static grad scaler [6135](https://github.com/Oneflow-Inc/oneflow/pull/6135)
- [**bug**][**eager**] Fix LayerNorm expr bug [6121](https://github.com/Oneflow-Inc/oneflow/pull/6121)
- [**bug**][**eager**][**api**] move numpy c api init in numpy.cpp, make np array contiguous before copying [6117](https://github.com/Oneflow-Inc/oneflow/pull/6117)
- [**enhancement**][**eager**][**refactor**] Remove params from ParamGroup getitem [6096](https://github.com/Oneflow-Inc/oneflow/pull/6096)
- [**enhancement**][**feature**][**eager**] Support tensor and optimizer serialization [6087](https://github.com/Oneflow-Inc/oneflow/pull/6087)
- [**enhancement**][**bug**][**eager**] fix bug about tensor str in nonsymmetric cast and getitem in consist… [6239](https://github.com/Oneflow-Inc/oneflow/pull/6239)
- [**enhancement**][**eager**] Cpu all reduce [5849](https://github.com/Oneflow-Inc/oneflow/pull/5849)
- [**feature**][**eager**] Support assign copy interface [6228](https://github.com/Oneflow-Inc/oneflow/pull/6228)
- [**enhancement**][**eager**][**api**][**interface**] Dev reconstruct pad ops [6223](https://github.com/Oneflow-Inc/oneflow/pull/6223)
- [**enhancement**][**eager**][**api**][**interface**] support flow.cuda.is_available [6124](https://github.com/Oneflow-Inc/oneflow/pull/6124)
- [**bug**][**eager**] make flow.\_C.local_all_reduce sync lanuched [6175](https://github.com/Oneflow-Inc/oneflow/pull/6175)
- [**enhancement**][**eager**] Rename flow to oneflow in user hint [6190](https://github.com/Oneflow-Inc/oneflow/pull/6190)
- [**bug**][**eager**][**tooling**][**test**][**api**][**interface**] Autotest generate input tensor [6206](https://github.com/Oneflow-Inc/oneflow/pull/6206)
- [**enhancement**][**eager**] consistent tensor zeros\_() [6202](https://github.com/Oneflow-Inc/oneflow/pull/6202)
- [**enhancement**][**eager**] Cpu mpi [5865](https://github.com/Oneflow-Inc/oneflow/pull/5865)

Build enhancements:

- [**bug**][**build**] Fix GRPC compilation failure on CMake 3.20 [5255](https://github.com/Oneflow-Inc/oneflow/pull/5255)
- [**bug**][**build**] Refine header file copy [5254](https://github.com/Oneflow-Inc/oneflow/pull/5254)
- [**bug**][**build**] Fix older version CMake doesn't support multiple targets in CLI [5248](https://github.com/Oneflow-Inc/oneflow/pull/5248)
- [**bug**][**build**] Turn off NCCL_STATIC/CUDNN_STATIC when CUDA_STATIC is OFF [5243](https://github.com/Oneflow-Inc/oneflow/pull/5243)
- [**feature**][**build**] Fix support for Ninja and add Ninja build in Simple CI [5236](https://github.com/Oneflow-Inc/oneflow/pull/5236)
- [**enhancement**][**build**] Add cmake option CUDA_STATIC [5164](https://github.com/Oneflow-Inc/oneflow/pull/5164)
- [**bug**][**build**] Fix protobuf debug postfix [5233](https://github.com/Oneflow-Inc/oneflow/pull/5233)
- [**enhancement**][**ci**][**build**] Move default third party dir into build dir [5230](https://github.com/Oneflow-Inc/oneflow/pull/5230)
- [**enhancement**][**build**] Refine protobuf cmake [5216](https://github.com/Oneflow-Inc/oneflow/pull/5216)
- [**enhancement**][**ci**][**build**] Remove transport test main [5215](https://github.com/Oneflow-Inc/oneflow/pull/5215)
- [**enhancement**][**ci**][**build**] Speedup opencv build [5213](https://github.com/Oneflow-Inc/oneflow/pull/5213)
- [**enhancement**][**build**] Support clang [5015](https://github.com/Oneflow-Inc/oneflow/pull/5015)
- [**enhancement**][**documentation**][**build**] Add prefix when creating git archive [5201](https://github.com/Oneflow-Inc/oneflow/pull/5201)
- [**enhancement**][**build**] Add cmake option NCCL_STATIC [5160](https://github.com/Oneflow-Inc/oneflow/pull/5160)
- [**enhancement**][**build**] Refine CMake CUDA version handling [5192](https://github.com/Oneflow-Inc/oneflow/pull/5192)
- [**enhancement**][**build**] Use clang plugin to check Maybe variables are used [5358](https://github.com/Oneflow-Inc/oneflow/pull/5358)
- [**enhancement**][**build**] Add BUILD_BYPRODUCTS for ExternalProject_Add [5316](https://github.com/Oneflow-Inc/oneflow/pull/5316)
- [**enhancement**][**build**] Add cmake init cache to simplify user onboarding [5311](https://github.com/Oneflow-Inc/oneflow/pull/5311)
- [**feature**][**bug**][**build**] Fix macOS support and run macOS build in Simple CI [4947](https://github.com/Oneflow-Inc/oneflow/pull/4947)
- [**enhancement**][**build**] flatbuffers use mirror [5295](https://github.com/Oneflow-Inc/oneflow/pull/5295)
- [**enhancement**][**build**] Don't build test by default [5302](https://github.com/Oneflow-Inc/oneflow/pull/5302)
- [**enhancement**][**build**] Prevent building from scratch when toggle flag BUILD_GIT_VERSION [5259](https://github.com/Oneflow-Inc/oneflow/pull/5259)
- [**enhancement**][**build**] Refine gRPC, glog, gflags cmake for conda [5276](https://github.com/Oneflow-Inc/oneflow/pull/5276)
- [**feature**][**build**] Support XLA with CPU-only [5260](https://github.com/Oneflow-Inc/oneflow/pull/5260)
- [**enhancement**][**ci**][**onnx**][**build**] Remove ONNX from CI [5257](https://github.com/Oneflow-Inc/oneflow/pull/5257)
- [**enhancement**][**build**] Refactor build_wheel to support oneflowinc images [5427](https://github.com/Oneflow-Inc/oneflow/pull/5427)
- [**enhancement**][**build**] Add arg skip_audit in build wheel [5423](https://github.com/Oneflow-Inc/oneflow/pull/5423)
- [**bug**][**build**] hwloc disable shared [5388](https://github.com/Oneflow-Inc/oneflow/pull/5388)
- [**documentation**][**build**] Update readme for autoconf and libtool [5376](https://github.com/Oneflow-Inc/oneflow/pull/5376)
- [**enhancement**][**build**] remove dir python and compatible_single_client_python [5609](https://github.com/Oneflow-Inc/oneflow/pull/5609)
- [**bug**][**build**][**system**] Fix pyyaml version [5594](https://github.com/Oneflow-Inc/oneflow/pull/5594)
- [**enhancement**][**ci**][**build**] force release flags [5574](https://github.com/Oneflow-Inc/oneflow/pull/5574)
- [**bug**][**build**] prevent endless loop [5534](https://github.com/Oneflow-Inc/oneflow/pull/5534)
- [**enhancement**][**build**] Support sccache [5528](https://github.com/Oneflow-Inc/oneflow/pull/5528)
- [**enhancement**][**build**] Add definition for CMAKE_BUILD_TYPE and print cmake_build_type in oneflow doctor [5505](https://github.com/Oneflow-Inc/oneflow/pull/5505)
- [**enhancement**][**ci**][**build**][**need-simple-ci**] Fix macOS for recent changes [5705](https://github.com/Oneflow-Inc/oneflow/pull/5705)
- [**bug**][**build**] fix return type error on gcc 4.8.5 [5660](https://github.com/Oneflow-Inc/oneflow/pull/5660)
- [**enhancement**][**build**] Check CMAKE_BUILD_TYPE [5656](https://github.com/Oneflow-Inc/oneflow/pull/5656)
- [**enhancement**][**build**] add -Werror=return-type [5655](https://github.com/Oneflow-Inc/oneflow/pull/5655)
- [**enhancement**][**build**] Clean and fix for new py dir [5618](https://github.com/Oneflow-Inc/oneflow/pull/5618)
- [**enhancement**][**build**] cmake: disable array-bounds check & treat warnings as errors for pyextobj and oneflow_internal & fix warnings [5838](https://github.com/Oneflow-Inc/oneflow/pull/5838)
- [**bug**][**build**] set CMAKE_BUILD_TYPE to Release if undefined [5842](https://github.com/Oneflow-Inc/oneflow/pull/5842)
- [**enhancement**][**build**][**need-simple-ci**] Fix all warnings & Add option TREAT_WARING_AS_ERROR to cmake [5751](https://github.com/Oneflow-Inc/oneflow/pull/5751)
- [**enhancement**][**build**] add CMAKE_INTERPROCEDURAL_OPTIMIZATION in fast cmake cache [5970](https://github.com/Oneflow-Inc/oneflow/pull/5970)
- [**enhancement**][**build**] add clang tidy target [5957](https://github.com/Oneflow-Inc/oneflow/pull/5957)
- [**bug**][**build**] cmake: fix cmake cache args in opencv [5959](https://github.com/Oneflow-Inc/oneflow/pull/5959)
- [**enhancement**][**build**] Add cmake option USE_SYSTEM_NCCL [5897](https://github.com/Oneflow-Inc/oneflow/pull/5897)
- [**enhancement**][**build**] cmake: include third party headers as system headers to avoid warnings [5879](https://github.com/Oneflow-Inc/oneflow/pull/5879)
- [**enhancement**][**build**] Ignore opencv-python on machine aarch64 [5884](https://github.com/Oneflow-Inc/oneflow/pull/5884)
- [**enhancement**][**build**] enable CMake first class cuda support [5858](https://github.com/Oneflow-Inc/oneflow/pull/5858)
- [**bug**][**build**] Fix compile warning (strict-aliasing) [5872](https://github.com/Oneflow-Inc/oneflow/pull/5872)
- [**enhancement**][**bug**][**build**][**need-simple-ci**] Upgrade gtest and fix some errors raised by clang [6079](https://github.com/Oneflow-Inc/oneflow/pull/6079)
- [**bug**][**ci**][**build**] cmake: fix ninja build in CI [6072](https://github.com/Oneflow-Inc/oneflow/pull/6072)
- [**bug**][**build**] fix files not actually removed when building for multiple python versions [6060](https://github.com/Oneflow-Inc/oneflow/pull/6060)
- [**bug**][**build**][**api**] functional_api: fix build error in mac os [6010](https://github.com/Oneflow-Inc/oneflow/pull/6010)
- [**bug**][**build**][**need-simple-ci**][**need-single-client-tests**] Fix recompile from scratch [6036](https://github.com/Oneflow-Inc/oneflow/pull/6036)
- [**bug**][**build**] Turn on NVCC's warnings [6011](https://github.com/Oneflow-Inc/oneflow/pull/6011)
- [**bug**][**build**][**need-single-client-tests**] fix bundle .so of other python version [6034](https://github.com/Oneflow-Inc/oneflow/pull/6034)
- [**bug**][**ci**][**build**][**need-single-client-tests**] use copy_all_files_in_dir to replace copy_files [6033](https://github.com/Oneflow-Inc/oneflow/pull/6033)
- [**enhancement**][**build**] check compiler version in cmake [6026](https://github.com/Oneflow-Inc/oneflow/pull/6026)
- [**enhancement**][**build**] Add CUDA_NVCC_THREADS_NUMBER [6017](https://github.com/Oneflow-Inc/oneflow/pull/6017)
- [**enhancement**][**build**][**need-simple-ci**] optimize of_include_copy [5978](https://github.com/Oneflow-Inc/oneflow/pull/5978)
- [**enhancement**][**ci**][**build**][**need-single-client-tests**] CI: remove `-DTREAT_WARNINGS_AS_ERRORS=OFF` [6008](https://github.com/Oneflow-Inc/oneflow/pull/6008)
- [**enhancement**][**build**][**xla**] xrt: fix all warnings [5915](https://github.com/Oneflow-Inc/oneflow/pull/5915)
- [**enhancement**][**build**] Prevent opencv compile failure with std 17 [5997](https://github.com/Oneflow-Inc/oneflow/pull/5997)
- [**enhancement**][**build**] Use bundled cub [5998](https://github.com/Oneflow-Inc/oneflow/pull/5998)
- [**enhancement**][**ci**][**build**] update clang tidy diff warnings-as-errors option [5989](https://github.com/Oneflow-Inc/oneflow/pull/5989)
- [**enhancement**][**build**] Update run_clang_tidy.py to set return code and add warning-as-errors [5977](https://github.com/Oneflow-Inc/oneflow/pull/5977)
- [**enhancement**][**build**] check: fix clang-tidy-diff commands [5972](https://github.com/Oneflow-Inc/oneflow/pull/5972)
- [**bug**][**build**] Suppress NVCC warning 177-D [6094](https://github.com/Oneflow-Inc/oneflow/pull/6094)

XLA enhancements:

- [**bug**][**xla**] Make the blob header memory aligned. [5286](https://github.com/Oneflow-Inc/oneflow/pull/5286)

System:

- [**enhancement**][**system**] Refactor Memory Zone [5072](https://github.com/Oneflow-Inc/oneflow/pull/5072)
- [**enhancement**][**system**] Add interface InferContext::OutputTensorDesc [5219](https://github.com/Oneflow-Inc/oneflow/pull/5219)
- [**bug**][**system**] Lazy construct functor to make sure that the operators has already been registered. [5225](https://github.com/Oneflow-Inc/oneflow/pull/5225)
- [**enhancement**][**system**] Refactor infer ctx output isdynamic [5220](https://github.com/Oneflow-Inc/oneflow/pull/5220)
- [**enhancement**][**system**] Refactor infer ctx input isdynamic [5211](https://github.com/Oneflow-Inc/oneflow/pull/5211)
- [**enhancement**][**system**] Wake up the heartbeat thread immediately [5081](https://github.com/Oneflow-Inc/oneflow/pull/5081)
- [**enhancement**][**system**] Fix xla test case fail [5203](https://github.com/Oneflow-Inc/oneflow/pull/5203)
- [**enhancement**][**system**] Add interface InferContext::InputDType [5153](https://github.com/Oneflow-Inc/oneflow/pull/5153)
- [**purge**][**system**] delete const_cast in Output [5196](https://github.com/Oneflow-Inc/oneflow/pull/5196)
- [**feature**][**system**] Add hwloc for topology detection [5291](https://github.com/Oneflow-Inc/oneflow/pull/5291)
- [**enhancement**][**system**] fix registry may segment [5336](https://github.com/Oneflow-Inc/oneflow/pull/5336)
- [**enhancement**][**system**] Use functional api instead of op_expr_helper::XXXOp. [5364](https://github.com/Oneflow-Inc/oneflow/pull/5364)
- [**enhancement**][**system**] move btob to op [5274](https://github.com/Oneflow-Inc/oneflow/pull/5274)
- [**documentation**][**system**] Add Latest News section in README [5361](https://github.com/Oneflow-Inc/oneflow/pull/5361)
- [**enhancement**][**bug**][**system**] fix dropout module: return directly if not training [5346](https://github.com/Oneflow-Inc/oneflow/pull/5346)
- [**bug**][**system**] add missing JUST [5357](https://github.com/Oneflow-Inc/oneflow/pull/5357)
- [**documentation**][**system**] Add more communication outlets on README [5359](https://github.com/Oneflow-Inc/oneflow/pull/5359)
- [**enhancement**][**feature**][**system**] CommNet dynamic register memory [5281](https://github.com/Oneflow-Inc/oneflow/pull/5281)
- [**enhancement**][**system**] Use symbol device [5341](https://github.com/Oneflow-Inc/oneflow/pull/5341)
- [**enhancement**][**system**] fix multithread bug in env [5283](https://github.com/Oneflow-Inc/oneflow/pull/5283)
- [**bug**][**system**][**api**] fix bug in cfg_replacement [5335](https://github.com/Oneflow-Inc/oneflow/pull/5335)
- [**bug**][**system**] Fix create log directory thread-unsafe [5326](https://github.com/Oneflow-Inc/oneflow/pull/5326)
- [**bug**][**system**] fix_bug_in_make_parallel [5328](https://github.com/Oneflow-Inc/oneflow/pull/5328)
- [**enhancement**][**system**][**cfg**] replace train_conf, job_conf using cfg::xx [5263](https://github.com/Oneflow-Inc/oneflow/pull/5263)
- [**enhancement**][**system**][**quantization**] support tensorrt in qat [5287](https://github.com/Oneflow-Inc/oneflow/pull/5287)
- [**enhancement**][**system**][**api**] Export functional apis for oneflow.experimental. [5313](https://github.com/Oneflow-Inc/oneflow/pull/5313)
- [**enhancement**][**system**] fix bug check between cfg enum and proto enum [5285](https://github.com/Oneflow-Inc/oneflow/pull/5285)
- [**enhancement**][**system**] replace CHECK_EQ using CHECK_EQ_OR_RETURN [5279](https://github.com/Oneflow-Inc/oneflow/pull/5279)
- [**enhancement**][**system**] Refactor SbpXXX to cfg::SbpXXX [5120](https://github.com/Oneflow-Inc/oneflow/pull/5120)
- [**enhancement**][**system**][**api**] add detach for LazyMirroredtensorImpl [5270](https://github.com/Oneflow-Inc/oneflow/pull/5270)
- [**enhancement**][**system**] shorten XXIsDynamic4ArgNameAndIndex to be xxIsDynamic [5265](https://github.com/Oneflow-Inc/oneflow/pull/5265)
- [**enhancement**][**system**][**cfg**] job_config to cfg [5235](https://github.com/Oneflow-Inc/oneflow/pull/5235)
- [**feature**][**system**] Multi-Client LogicalRun degenerate to PhysicalRun [5479](https://github.com/Oneflow-Inc/oneflow/pull/5479)
- [**enhancement**][**system**] fix ConstructOp without JUST [5480](https://github.com/Oneflow-Inc/oneflow/pull/5480)
- [**enhancement**][**system**] Output arg modifier return maybe part 1 [5451](https://github.com/Oneflow-Inc/oneflow/pull/5451)
- [**feature**][**system**][**interface**] Fea/nn graph/graph build ctx [5420](https://github.com/Oneflow-Inc/oneflow/pull/5420)
- [**enhancement**][**system**] Throw exception if check failed [5457](https://github.com/Oneflow-Inc/oneflow/pull/5457)
- [**feature**][**system**] multi client launch [5372](https://github.com/Oneflow-Inc/oneflow/pull/5372)
- [**enhancement**][**system**][**api**] Optimize reduce mean [5452](https://github.com/Oneflow-Inc/oneflow/pull/5452)
- [**enhancement**][**system**] export Tensor only to python [5440](https://github.com/Oneflow-Inc/oneflow/pull/5440)
- [**enhancement**][**system**] Output arg modifier return maybe part_0 [5447](https://github.com/Oneflow-Inc/oneflow/pull/5447)
- [**enhancement**][**system**] ThreadMgr support AddPlan [5450](https://github.com/Oneflow-Inc/oneflow/pull/5450)
- [**enhancement**][**system**] Refactor infer ctx input tensordesc [5226](https://github.com/Oneflow-Inc/oneflow/pull/5226)
- [**enhancement**][**system**][**api**] instruction builder return maybe [5442](https://github.com/Oneflow-Inc/oneflow/pull/5442)
- [**feature**][**system**][**interface**] MultiClientSessionContext [5421](https://github.com/Oneflow-Inc/oneflow/pull/5421)
- [**enhancement**][**feature**][**system**] add launcher, update multi client launch and exit [5414](https://github.com/Oneflow-Inc/oneflow/pull/5414)
- [**purge**][**system**][**refactor**] Remove IOConf [5419](https://github.com/Oneflow-Inc/oneflow/pull/5419)
- [**enhancement**][**system**] Dev refine generator [5426](https://github.com/Oneflow-Inc/oneflow/pull/5426)
- [**enhancement**][**system**] Support inplace operations [5204](https://github.com/Oneflow-Inc/oneflow/pull/5204)
- [**enhancement**][**system**][**refactor**] Dev refactor generator [5397](https://github.com/Oneflow-Inc/oneflow/pull/5397)
- [**enhancement**][**system**] Add new placement init func [5408](https://github.com/Oneflow-Inc/oneflow/pull/5408)
- [**enhancement**][**system**] NNGraphIf [5387](https://github.com/Oneflow-Inc/oneflow/pull/5387)
- [**enhancement**][**system**][**refactor**] Cast explicitily in unpack call to avoid confilt with Optional. [5380](https://github.com/Oneflow-Inc/oneflow/pull/5380)
- [**enhancement**][**system**][**interface**] [Random Generator] Part2: Migrate functional dropout [5378](https://github.com/Oneflow-Inc/oneflow/pull/5378)
- [**enhancement**][**system**] replace ForeignJobInstance using JobInstance [5374](https://github.com/Oneflow-Inc/oneflow/pull/5374)
- [**enhancement**][**system**][**refactor**] Speedup reshape module by 5x. [5381](https://github.com/Oneflow-Inc/oneflow/pull/5381)
- [**feature**][**system**][**interface**] [Random Generator] Part1: Dev random generator [5360](https://github.com/Oneflow-Inc/oneflow/pull/5360)
- [**enhancement**][**system**] Add ONEFLOW_STREAM_CUDA_EVENT_FLAG_BLOCKING_SYNC [5612](https://github.com/Oneflow-Inc/oneflow/pull/5612)
- [**enhancement**][**system**] [part2]Remove singleclient outdated api [5568](https://github.com/Oneflow-Inc/oneflow/pull/5568)
- [**feature**][**system**][**interface**] nn.Graph call and launch impl [5580](https://github.com/Oneflow-Inc/oneflow/pull/5580)
- [**enhancement**][**system**] remove outdated doctest api and "experimental_api" [5564](https://github.com/Oneflow-Inc/oneflow/pull/5564)
- [**feature**][**system**][**interface**] Register ForeignCallback and Watcher in Multi-Client [5591](https://github.com/Oneflow-Inc/oneflow/pull/5591)
- [**enhancement**][**system**] [Part-1]remove outdated api and files of multi-client on master branch [5556](https://github.com/Oneflow-Inc/oneflow/pull/5556)
- [**feature**][**system**][**interface**] LazyInterpret build LocalTensor if input is local [5582](https://github.com/Oneflow-Inc/oneflow/pull/5582)
- [**enhancement**][**system**] add job_pass MultiClientAutoSourceAndSinkTick [5507](https://github.com/Oneflow-Inc/oneflow/pull/5507)
- [**feature**][**system**] Fea/nn graph/optimizer [5533](https://github.com/Oneflow-Inc/oneflow/pull/5533)
- [**feature**][**system**][**interface**] New/CloseRuntimeBuffers and RunLazyJob impl [5571](https://github.com/Oneflow-Inc/oneflow/pull/5571)
- [**feature**][**system**][**refactor**][**interface**] NNGraph interface and implement for CompileAndRuntime [5558](https://github.com/Oneflow-Inc/oneflow/pull/5558)
- [**feature**][**system**] Fea/nn graph/forward graph [5516](https://github.com/Oneflow-Inc/oneflow/pull/5516)
- [**enhancement**][**system**] Lazy job stream type [5389](https://github.com/Oneflow-Inc/oneflow/pull/5389)
- [**enhancement**][**system**] Refactor single client autotick [5506](https://github.com/Oneflow-Inc/oneflow/pull/5506)
- [**enhancement**][**system**] replace underline using dot in single client [5547](https://github.com/Oneflow-Inc/oneflow/pull/5547)
- [**bug**][**system**] fix return type [5548](https://github.com/Oneflow-Inc/oneflow/pull/5548)
- [**feature**][**system**][**interface**] LazyInterpret for UserOpExpr [5544](https://github.com/Oneflow-Inc/oneflow/pull/5544)
- [**enhancement**][**system**] Add ProfilerStart/ProfilerStop API [5542](https://github.com/Oneflow-Inc/oneflow/pull/5542)
- [**feature**][**system**][**interface**] LazyInterpreter for FetchOutputOpExpr and set op parallel_distribution [5527](https://github.com/Oneflow-Inc/oneflow/pull/5527)
- [**enhancement**][**system**] Multi client push pull [5492](https://github.com/Oneflow-Inc/oneflow/pull/5492)
- [**enhancement**][**system**] registry_callback_fn return maybe [5456](https://github.com/Oneflow-Inc/oneflow/pull/5456)
- [**enhancement**][**system**] bw_gen_fn return maybe [5455](https://github.com/Oneflow-Inc/oneflow/pull/5455)
- [**enhancement**][**system**] gen_bw_fn return maybe [5454](https://github.com/Oneflow-Inc/oneflow/pull/5454)
- [**enhancement**][**system**] Compatible single client [5417](https://github.com/Oneflow-Inc/oneflow/pull/5417)
- [**feature**][**system**][**interface**] GlobalMultiClientEnv and refine EagerExecution [5523](https://github.com/Oneflow-Inc/oneflow/pull/5523)
- [**enhancement**][**system**] Job pass maybe system [5503](https://github.com/Oneflow-Inc/oneflow/pull/5503)
- [**enhancement**][**system**] Remove Plan::net_topo [5502](https://github.com/Oneflow-Inc/oneflow/pull/5502)
- [**feature**][**system**][**interface**] LazyInterpret for FeedVariableOpExpr [5490](https://github.com/Oneflow-Inc/oneflow/pull/5490)
- [**enhancement**][**system**] Input arg modifier return maybe [5453](https://github.com/Oneflow-Inc/oneflow/pull/5453)
- [**feature**][**system**][**interface**] Fea/nn graph/block scope [5498](https://github.com/Oneflow-Inc/oneflow/pull/5498)
- [**feature**][**system**] jit_fuse_cast_scale [5332](https://github.com/Oneflow-Inc/oneflow/pull/5332)
- [**enhancement**][**system**] Remove obsolete Profiler [5747](https://github.com/Oneflow-Inc/oneflow/pull/5747)
- [**enhancement**][**system**][**api**] Dev fix batch norm not stats [5733](https://github.com/Oneflow-Inc/oneflow/pull/5733)
- [**enhancement**][**system**] rename rpc_token to TransportToken [5735](https://github.com/Oneflow-Inc/oneflow/pull/5735)
- [**enhancement**][**system**][**api**] Refacotr maximum minimum py2cpp [5724](https://github.com/Oneflow-Inc/oneflow/pull/5724)
- [**enhancement**][**system**] Replace piece_id with comm_net_sequence_number [5731](https://github.com/Oneflow-Inc/oneflow/pull/5731)
- [**enhancement**][**system**] beautify stack frame [5686](https://github.com/Oneflow-Inc/oneflow/pull/5686)
- [**enhancement**][**system**] Add env ONEFLOW_KERNEL_DISABLE_BLOB_ACCESS_CHECKER [5728](https://github.com/Oneflow-Inc/oneflow/pull/5728)
- [**enhancement**][**system**] Add env ONEFLOW_THREAD_ENABLE_LOCAL_MESSAGE_QUEUE [5720](https://github.com/Oneflow-Inc/oneflow/pull/5720)
- [**enhancement**][**system**][**api**][**refactor**] Refactor functional sub, mul and div apis [5713](https://github.com/Oneflow-Inc/oneflow/pull/5713)
- [**feature**][**system**] ddp [5008](https://github.com/Oneflow-Inc/oneflow/pull/5008)
- [**enhancement**][**system**][**api**][**refactor**] Refactor functional matmul and add apis. [5697](https://github.com/Oneflow-Inc/oneflow/pull/5697)
- [**bug**][**system**] Fix ClearKV("plan") [5710](https://github.com/Oneflow-Inc/oneflow/pull/5710)
- [**enhancement**][**system**] Rename cpu to async cpu [5712](https://github.com/Oneflow-Inc/oneflow/pull/5712)
- [**enhancement**][**system**] Support tensor.to()/to_local() [5271](https://github.com/Oneflow-Inc/oneflow/pull/5271)
- [**feature**][**system**][**refactor**][**interface**] Multi-Runtime for multi nn.Graph [5683](https://github.com/Oneflow-Inc/oneflow/pull/5683)
- [**bug**][**system**][**refactor**] Add tag for Optional inplace constructor [5619](https://github.com/Oneflow-Inc/oneflow/pull/5619)
- [**enhancement**][**system**] Move Global<CommNet> to env scope [5670](https://github.com/Oneflow-Inc/oneflow/pull/5670)
- [**enhancement**][**system**] add JUST wrapper [5681](https://github.com/Oneflow-Inc/oneflow/pull/5681)
- [**enhancement**][**system**] New sync consistent meta info [5634](https://github.com/Oneflow-Inc/oneflow/pull/5634)
- [**enhancement**][**system**][**refactor**][**interface**] Refactor RuntimeCtx for multi-runtime [5664](https://github.com/Oneflow-Inc/oneflow/pull/5664)
- [**feature**][**system**][**interface**] Feat: memory shared between EagerTensor with VariableRegst [5649](https://github.com/Oneflow-Inc/oneflow/pull/5649)
- [**enhancement**][**system**] Use functional call directly instead of construct a module and then call-Add [5613](https://github.com/Oneflow-Inc/oneflow/pull/5613)
- [**enhancement**][**system**] disable eager_op consistent mode [5647](https://github.com/Oneflow-Inc/oneflow/pull/5647)
- [**enhancement**][**system**] add msg_penddin_list in ibverbs_qp to optimize qp_init_attr.cap.max_send_wr [5485](https://github.com/Oneflow-Inc/oneflow/pull/5485)
- [**enhancement**][**system**] IBVerbsCommNet add knobs [5626](https://github.com/Oneflow-Inc/oneflow/pull/5626)
- [**enhancement**][**system**] Prune python tensor [5596](https://github.com/Oneflow-Inc/oneflow/pull/5596)
- [**feature**][**system**][**interface**] Feat: LazyInterpret infer op / tensor ParallelDescScope [5625](https://github.com/Oneflow-Inc/oneflow/pull/5625)
- [**enhancement**][**system**] Replace src tick with with wait and send ids [5603](https://github.com/Oneflow-Inc/oneflow/pull/5603)
- [**enhancement**][**system**] Support symbol placement type in functional. [5627](https://github.com/Oneflow-Inc/oneflow/pull/5627)
- [**enhancement**][**system**][**api**][**refactor**][**interface**] Dev advanced indexing [5559](https://github.com/Oneflow-Inc/oneflow/pull/5559)
- [**enhancement**][**system**] Optimize maybe. [5839](https://github.com/Oneflow-Inc/oneflow/pull/5839)
- [**enhancement**][**system**] Decorator 4 disable recursive boxing call [5796](https://github.com/Oneflow-Inc/oneflow/pull/5796)
- [**enhancement**][**system**] add_eager_boxing_and_op_interpreter_dispatch_error_info [5819](https://github.com/Oneflow-Inc/oneflow/pull/5819)
- [**enhancement**][**system**] Kernel CUDA Graphs Support [5725](https://github.com/Oneflow-Inc/oneflow/pull/5725)
- [**bug**][**system**] Fix placement print bug [5853](https://github.com/Oneflow-Inc/oneflow/pull/5853)
- [**bug**][**system**] when error msg formatting fails, return error->DebugString [5844](https://github.com/Oneflow-Inc/oneflow/pull/5844)
- [**enhancement**][**system**][**refactor**] Rename variables named `*parallel_distribution*` to `*nd_sbp*` (1) [5815](https://github.com/Oneflow-Inc/oneflow/pull/5815)
- [**feature**][**system**][**interface**] Support Free EagerTensor caught in nn.Graph build [5777](https://github.com/Oneflow-Inc/oneflow/pull/5777)
- [**enhancement**][**system**] Reuse CUDA event / Refine BnInOp2Blob / Refine channel [5837](https://github.com/Oneflow-Inc/oneflow/pull/5837)
- [**enhancement**][**system**][**serving**] fix bug in AddInputOutputOpsPass: check existence of key in HashMap(inferface_lbi2scope_sym_id) [5653](https://github.com/Oneflow-Inc/oneflow/pull/5653)
- [**enhancement**][**system**][**api**] unpack_call: impl new `unpack_call_dispatcher` for better performance [5820](https://github.com/Oneflow-Inc/oneflow/pull/5820)
- [**feature**][**system**] Feat consistent tensor python constructor [5812](https://github.com/Oneflow-Inc/oneflow/pull/5812)
- [**feature**][**system**] Support 0shape tensor [5620](https://github.com/Oneflow-Inc/oneflow/pull/5620)
- [**documentation**][**system**] fix launcher description [5770](https://github.com/Oneflow-Inc/oneflow/pull/5770)
- [**feature**][**system**][**interface**] Multi-nn.Graph memory reuse by Chunk manager [5658](https://github.com/Oneflow-Inc/oneflow/pull/5658)
- [**bug**][**system**] Fix naive b2p error [5806](https://github.com/Oneflow-Inc/oneflow/pull/5806)
- [**enhancement**][**system**] set created generator with default rng seed [5801](https://github.com/Oneflow-Inc/oneflow/pull/5801)
- [**enhancement**][**system**] enhance_local_to_consistent [5761](https://github.com/Oneflow-Inc/oneflow/pull/5761)
- [**feature**][**system**] add flow.randn [5736](https://github.com/Oneflow-Inc/oneflow/pull/5736)
- [**enhancement**][**system**] Refactor hierarchical parallel cast autograd [5764](https://github.com/Oneflow-Inc/oneflow/pull/5764)
- [**enhancement**][**system**] Collective boxing executor add_plan delete_plan [5495](https://github.com/Oneflow-Inc/oneflow/pull/5495)
- [**enhancement**][**system**] Fix throw abort [5795](https://github.com/Oneflow-Inc/oneflow/pull/5795)
- [**enhancement**][**system**] DECORATE [5794](https://github.com/Oneflow-Inc/oneflow/pull/5794)
- [**enhancement**][**system**] Inferface eager boxing [5682](https://github.com/Oneflow-Inc/oneflow/pull/5682)
- [**enhancement**][**system**] extract_consistent_to_consistent_op_expr [5870](https://github.com/Oneflow-Inc/oneflow/pull/5870)
- [**enhancement**][**system**] disable backward pass consistent tensor meta check. [5871](https://github.com/Oneflow-Inc/oneflow/pull/5871)
- [**enhancement**][**system**] Add CudaStreamIndexGenerator::GenerateNamedStreamIndex [5940](https://github.com/Oneflow-Inc/oneflow/pull/5940)
- [**bug**][**system**] Only query PCI bus id when CUDA version >= 11 [5937](https://github.com/Oneflow-Inc/oneflow/pull/5937)
- [**enhancement**][**system**] maybe: add `JUST_MSG` and `CHECK_JUST_MSG` [5904](https://github.com/Oneflow-Inc/oneflow/pull/5904)
- [**bug**][**system**] Fix bug scalar [5950](https://github.com/Oneflow-Inc/oneflow/pull/5950)
- [**enhancement**][**system**] framework: fix rvalue reference warnings [5948](https://github.com/Oneflow-Inc/oneflow/pull/5948)
- [**purge**][**system**] Remove CudaWorkType [5942](https://github.com/Oneflow-Inc/oneflow/pull/5942)
- [**enhancement**][**system**] refactor_symbol [5941](https://github.com/Oneflow-Inc/oneflow/pull/5941)
- [**bug**][**system**] consistent_tensor_infer_cache: fix memory leak [5938](https://github.com/Oneflow-Inc/oneflow/pull/5938)
- [**feature**][**system**] support to print gpu [5936](https://github.com/Oneflow-Inc/oneflow/pull/5936)
- [**enhancement**][**system**] Bugfix static check [5935](https://github.com/Oneflow-Inc/oneflow/pull/5935)
- [**bug**][**system**] fix nccl_version log [5934](https://github.com/Oneflow-Inc/oneflow/pull/5934)
- [**bug**][**system**] Fix bug of multi-GPU train nn.Graph extra mem cost in rank 0 [5930](https://github.com/Oneflow-Inc/oneflow/pull/5930)
- [**enhancement**][**system**] Only gradient acc be scheduled in parallel. [5926](https://github.com/Oneflow-Inc/oneflow/pull/5926)
- [**enhancement**][**bug**][**system**] fix_ddp_bug_on_8_process [5929](https://github.com/Oneflow-Inc/oneflow/pull/5929)
- [**enhancement**][**system**] Fix bug error msg format [5866](https://github.com/Oneflow-Inc/oneflow/pull/5866)
- [**feature**][**system**] print consistent tensor data [5902](https://github.com/Oneflow-Inc/oneflow/pull/5902)
- [**bug**][**system**] Move parse env to the constructor [5922](https://github.com/Oneflow-Inc/oneflow/pull/5922)
- [**enhancement**][**system**] Remove GlobalWorkStreamId/GlobalThrdId [5917](https://github.com/Oneflow-Inc/oneflow/pull/5917)
- [**bug**][**system**] shared_or_scalar: fix alias warnings [5916](https://github.com/Oneflow-Inc/oneflow/pull/5916)
- [**purge**][**system**] Remove CompActor [5919](https://github.com/Oneflow-Inc/oneflow/pull/5919)
- [**enhancement**][**system**] Use symbol dtype [5641](https://github.com/Oneflow-Inc/oneflow/pull/5641)
- [**enhancement**][**feature**][**system**] Control Graph / Session / Env's python c++ object destruction [5845](https://github.com/Oneflow-Inc/oneflow/pull/5845)
- [**enhancement**][**bug**][**system**] Sync access and assign indexing tensor. [5907](https://github.com/Oneflow-Inc/oneflow/pull/5907)
- [**enhancement**][**system**][**api**][**refactor**] Dev consistent arange [5883](https://github.com/Oneflow-Inc/oneflow/pull/5883)
- [**enhancement**][**system**] Lazy interpreter for new ConsistentToConsistentOpExpr [5903](https://github.com/Oneflow-Inc/oneflow/pull/5903)
- [**bug**][**system**] Fix BUG of LazyInterpret FreeEagerTensor memory shared with regst [5891](https://github.com/Oneflow-Inc/oneflow/pull/5891)
- [**bug**][**system**] fix typo in `raise RuntimeError` [5890](https://github.com/Oneflow-Inc/oneflow/pull/5890)
- [**enhancement**][**system**][**refactor**] Rename the `ParallelDistribution` class to `NdSbp` [5814](https://github.com/Oneflow-Inc/oneflow/pull/5814)
- [**feature**][**system**] add flow.rand [5722](https://github.com/Oneflow-Inc/oneflow/pull/5722)
- [**feature**][**system**] Lazy Interpret support infer default device cpu [5880](https://github.com/Oneflow-Inc/oneflow/pull/5880)
- [**enhancement**][**system**] Tensor str [5783](https://github.com/Oneflow-Inc/oneflow/pull/5783)
- [**feature**][**system**][**interface**] Lazy to_consistent [5774](https://github.com/Oneflow-Inc/oneflow/pull/5774)
- [**enhancement**][**system**] wait vm empty before exiting [5860](https://github.com/Oneflow-Inc/oneflow/pull/5860)
- [**enhancement**][**system**] Eager boxing n to 1 [5949](https://github.com/Oneflow-Inc/oneflow/pull/5949)
- [**enhancement**][**system**] add kernel observer [6052](https://github.com/Oneflow-Inc/oneflow/pull/6052)
- [**enhancement**][**ci**][**system**] Optimize ddp broadcast and add speed/memory test in ci [6044](https://github.com/Oneflow-Inc/oneflow/pull/6044)
- [**enhancement**][**system**] add var to control only print warning once when blocked [6045](https://github.com/Oneflow-Inc/oneflow/pull/6045)
- [**enhancement**][**system**][**refactor**] Rewrite pow and logical functional apis [6032](https://github.com/Oneflow-Inc/oneflow/pull/6032)
- [**enhancement**][**system**] Token seq id [5964](https://github.com/Oneflow-Inc/oneflow/pull/5964)
- [**enhancement**][**documentation**][**system**] Remove python function wrapper. [6012](https://github.com/Oneflow-Inc/oneflow/pull/6012)
- [**feature**][**system**] Add timeout and loc for blocking calls [6007](https://github.com/Oneflow-Inc/oneflow/pull/6007)
- [**enhancement**][**system**] Eager boxing 1 to n [5943](https://github.com/Oneflow-Inc/oneflow/pull/5943)
- [**enhancement**][**system**] Boxing expr [6015](https://github.com/Oneflow-Inc/oneflow/pull/6015)
- [**enhancement**][**system**] new_X_to_B [5987](https://github.com/Oneflow-Inc/oneflow/pull/5987)
- [**enhancement**][**system**] Add unimplemented return information [5952](https://github.com/Oneflow-Inc/oneflow/pull/5952)
- [**enhancement**][**system**] Revert "Faster decorator" [6006](https://github.com/Oneflow-Inc/oneflow/pull/6006)
- [**enhancement**][**system**] Throw exception if using advanced indexing for tensor setitem [6001](https://github.com/Oneflow-Inc/oneflow/pull/6001)
- [**enhancement**][**system**] Support eager boxing sm 2 sn [5869](https://github.com/Oneflow-Inc/oneflow/pull/5869)
- [**enhancement**][**system**] Move framework/local_dep_object.\* to the eager directory [5988](https://github.com/Oneflow-Inc/oneflow/pull/5988)
- [**enhancement**][**system**] Fix builtin op arg tuple. [5464](https://github.com/Oneflow-Inc/oneflow/pull/5464)
- [**feature**][**system**][**refactor**] Dev functional multiple signatures [5982](https://github.com/Oneflow-Inc/oneflow/pull/5982)
- [**enhancement**][**system**] Faster decorator [5996](https://github.com/Oneflow-Inc/oneflow/pull/5996)
- [**enhancement**][**system**] Placed nd sbp [5995](https://github.com/Oneflow-Inc/oneflow/pull/5995)
- [**feature**][**system**] Support asymmetric input/output/variable tensors in nn.Graph [5983](https://github.com/Oneflow-Inc/oneflow/pull/5983)
- [**enhancement**][**system**] LightActor [5868](https://github.com/Oneflow-Inc/oneflow/pull/5868)
- [**bug**][**system**] Prevent running oneflow in forked subprocess [5976](https://github.com/Oneflow-Inc/oneflow/pull/5976)
- [**bug**][**system**] common/error: fix build error in mac os [5971](https://github.com/Oneflow-Inc/oneflow/pull/5971)
- [**bug**][**system**] fix_bug_test_tensor_str [5958](https://github.com/Oneflow-Inc/oneflow/pull/5958)
- [**enhancement**][**system**] Refine StreamContext [6191](https://github.com/Oneflow-Inc/oneflow/pull/6191)
- [**enhancement**][**system**] container_util: fix VectorAt, remove useless MutMapAt [6172](https://github.com/Oneflow-Inc/oneflow/pull/6172)
- [**enhancement**][**system**] Typesafe KernelState [6198](https://github.com/Oneflow-Inc/oneflow/pull/6198)
- [**enhancement**][**system**] Primitive based copy task node [6195](https://github.com/Oneflow-Inc/oneflow/pull/6195)
- [**feature**][**system**][**interface**] Lazy support Scalar [6181](https://github.com/Oneflow-Inc/oneflow/pull/6181)
- [**enhancement**][**system**] Disable implicit boxing when parallel num eq one [6188](https://github.com/Oneflow-Inc/oneflow/pull/6188)
- [**enhancement**][**system**] Primitive [6183](https://github.com/Oneflow-Inc/oneflow/pull/6183)
- [**enhancement**][**system**] Remove IDMgr::GetGpuPhyIdFromThrdId/IDMgr::GetDeviceTypeFromThrdId [6169](https://github.com/Oneflow-Inc/oneflow/pull/6169)
- [**enhancement**][**system**] remove op_expr_helper inside gradient_funcs [6057](https://github.com/Oneflow-Inc/oneflow/pull/6057)
- [**feature**][**system**][**api**] Add tensor yaml, support export tensor functional api. [6099](https://github.com/Oneflow-Inc/oneflow/pull/6099)
- [**feature**][**system**] Plan memory log [6151](https://github.com/Oneflow-Inc/oneflow/pull/6151)
- [**feature**][**system**] Add dtype bfloat16 [5304](https://github.com/Oneflow-Inc/oneflow/pull/5304)
- [**enhancement**][**system**] StreamContext [6129](https://github.com/Oneflow-Inc/oneflow/pull/6129)
- [**bug**][**system**] Fix wrong inplace acc grad [6146](https://github.com/Oneflow-Inc/oneflow/pull/6146)
- [**enhancement**][**system**] UserKernel remove job_desc [6144](https://github.com/Oneflow-Inc/oneflow/pull/6144)
- [**enhancement**][**system**][**api**] Fea/graph/add outputs buffer to enable pipeline [6126](https://github.com/Oneflow-Inc/oneflow/pull/6126)
- [**enhancement**][**system**] not fuse request for nccl 2.10.3 [6136](https://github.com/Oneflow-Inc/oneflow/pull/6136)
- [**bug**][**system**] NewUniqueId thread safe [6141](https://github.com/Oneflow-Inc/oneflow/pull/6141)
- [**enhancement**][**system**] XRT remove job_desc [6139](https://github.com/Oneflow-Inc/oneflow/pull/6139)
- [**enhancement**][**system**] SystemOpFillJobNamePass [6138](https://github.com/Oneflow-Inc/oneflow/pull/6138)
- [**enhancement**][**system**] mv_boxing_folder_to_core [6140](https://github.com/Oneflow-Inc/oneflow/pull/6140)
- [**enhancement**][**system**] Refactor boxing interpreter to boxing expr [6134](https://github.com/Oneflow-Inc/oneflow/pull/6134)
- [**enhancement**][**system**] Eager boxing one to one [6048](https://github.com/Oneflow-Inc/oneflow/pull/6048)
- [**enhancement**][**system**] Vm cpu efficiency [6110](https://github.com/Oneflow-Inc/oneflow/pull/6110)
- [**enhancement**][**system**] Naive generic boxing [6116](https://github.com/Oneflow-Inc/oneflow/pull/6116)
- [**feature**][**system**] send/recv [5992](https://github.com/Oneflow-Inc/oneflow/pull/5992)
- [**enhancement**][**system**] disable_print_stack_in_tensor_numpy [6123](https://github.com/Oneflow-Inc/oneflow/pull/6123)
- [**feature**][**system**] add all_reduce by to_consistent [5963](https://github.com/Oneflow-Inc/oneflow/pull/5963)
- [**enhancement**][**system**] KernelContext [6084](https://github.com/Oneflow-Inc/oneflow/pull/6084)
- [**enhancement**][**bug**][**system**] Fix sync nccl and async nccl deadlock [6071](https://github.com/Oneflow-Inc/oneflow/pull/6071)
- [**bug**][**system**][**refactor**] Refactor to local [6098](https://github.com/Oneflow-Inc/oneflow/pull/6098)
- [**enhancement**][**system**] Replace xor with hash combine (part 1) [6078](https://github.com/Oneflow-Inc/oneflow/pull/6078)
- [**enhancement**][**system**] Optimize error message [6073](https://github.com/Oneflow-Inc/oneflow/pull/6073)
- [**enhancement**][**system**] Rename Error::xx to Error::xxError [6049](https://github.com/Oneflow-Inc/oneflow/pull/6049)
- [**enhancement**][**system**] send formatted msg to glog [5999](https://github.com/Oneflow-Inc/oneflow/pull/5999)
- [**feature**][**bottleneck**][**bug**][**system**][**interface**] [Feat.] NNGraph new eager tensor for new variable created in JobPass [6091](https://github.com/Oneflow-Inc/oneflow/pull/6091)
- [**bug**][**system**] Fix bug of multi-GPU eager copy D2H extra mem cost in rank 0 [6092](https://github.com/Oneflow-Inc/oneflow/pull/6092)
- [**enhancement**][**system**][**api**] Rename module flow.F to flow.\_C [6053](https://github.com/Oneflow-Inc/oneflow/pull/6053)
- [**feature**][**system**][**interface**] [Feat.] Eager consistent OFRecordReader [6089](https://github.com/Oneflow-Inc/oneflow/pull/6089)
- [**enhancement**][**system**][**api**] Dev fix and align interface [6075](https://github.com/Oneflow-Inc/oneflow/pull/6075)
- [**feature**][**bottleneck**][**bug**][**system**][**interface**] NNGraph input/output valid by register tensors [6240](https://github.com/Oneflow-Inc/oneflow/pull/6240)
- [**bug**][**system**][**interface**] Fix bug of Multi-Client src tick output order [6221](https://github.com/Oneflow-Inc/oneflow/pull/6221)
- [**enhancement**][**bug**][**system**] Add cast primitive [6234](https://github.com/Oneflow-Inc/oneflow/pull/6234)
- [**feature**][**bottleneck**][**system**][**interface**] Auto FixPipelineStageIdPass [6204](https://github.com/Oneflow-Inc/oneflow/pull/6204)
- [**enhancement**][**system**] move scalar to oneflow namespace. [6235](https://github.com/Oneflow-Inc/oneflow/pull/6235)
- [**enhancement**][**system**] UserKernel init CUDA Graphs with state [6230](https://github.com/Oneflow-Inc/oneflow/pull/6230)
- [**feature**][**system**] Comm broadcast [6213](https://github.com/Oneflow-Inc/oneflow/pull/6213)
- [**enhancement**][**system**][**refactor**] Rename op*name* to op*type_name* in AutogradEngine [6154](https://github.com/Oneflow-Inc/oneflow/pull/6154)
- [**enhancement**][**system**] Add memset primitive [6218](https://github.com/Oneflow-Inc/oneflow/pull/6218)
- [**enhancement**][**system**] Add StreamContext::device_type()/DeviceCtx::device_type() [6217](https://github.com/Oneflow-Inc/oneflow/pull/6217)
- [**feature**][**system**] add all_gather and fix bug of multi rank doctest [6189](https://github.com/Oneflow-Inc/oneflow/pull/6189)
- [**feature**][**system**][**interface**] [Feat.] Lazy interpreter skip hierarchical_parallel_cast [6208](https://github.com/Oneflow-Inc/oneflow/pull/6208)
- [**purge**][**system**] Cleanup KernelUtil [6212](https://github.com/Oneflow-Inc/oneflow/pull/6212)
- [**enhancement**][**system**] StreamContextAdapter [6205](https://github.com/Oneflow-Inc/oneflow/pull/6205)
- [**enhancement**][**system**] Dev eliminate gcc warnings [6199](https://github.com/Oneflow-Inc/oneflow/pull/6199)
- [**feature**][**bottleneck**][**system**][**interface**] [Feat.] nn.Graph support grad acc with input/output tensor [6155](https://github.com/Oneflow-Inc/oneflow/pull/6155)
- [**enhancement**][**system**] Cpu symetric s to s [6153](https://github.com/Oneflow-Inc/oneflow/pull/6153)
- [**enhancement**][**system**][**upload-core**] Op expr infer tensor meta [5064](https://github.com/Oneflow-Inc/oneflow/pull/5064)
- [**enhancement**][**system**] Infer consistent tensor meta [5362](https://github.com/Oneflow-Inc/oneflow/pull/5362)

CI enhancements:

- [**bug**][**ci**][**api**][**interface**] Refine module test [5232](https://github.com/Oneflow-Inc/oneflow/pull/5232)
- [**enhancement**][**ci**] Add Simple CI, runs CPU-only on GitHub hosted servers [5207](https://github.com/Oneflow-Inc/oneflow/pull/5207)
- [**enhancement**][**ci**] Run exe test in CPU-only [5202](https://github.com/Oneflow-Inc/oneflow/pull/5202)
- [**enhancement**][**ci**] Cancel all workflow runs but the latest [5206](https://github.com/Oneflow-Inc/oneflow/pull/5206)
- [**enhancement**][**ci**] Fix master not running Simple CI [5368](https://github.com/Oneflow-Inc/oneflow/pull/5368)
- [**enhancement**][**ci**] Refine Simple CI and Clang analysis [5367](https://github.com/Oneflow-Inc/oneflow/pull/5367)
- [**enhancement**][**feature**][**bug**][**ci**][**documentation**][**interface**] Fix upsample bilinear bug [5363](https://github.com/Oneflow-Inc/oneflow/pull/5363)
- [**enhancement**][**ci**] Build nightly for py39 [5318](https://github.com/Oneflow-Inc/oneflow/pull/5318)
- [**enhancement**][**ci**] Try distributed run for 3 times to prevent failure [5305](https://github.com/Oneflow-Inc/oneflow/pull/5305)
- [**enhancement**][**ci**] Upload Simple CI logs to cloud [5268](https://github.com/Oneflow-Inc/oneflow/pull/5268)
- [**enhancement**][**ci**] Remove cpu_op_eager and cuda_op_eager [5470](https://github.com/Oneflow-Inc/oneflow/pull/5470)
- [**bug**][**ci**] fix segfault in clang plugin [5437](https://github.com/Oneflow-Inc/oneflow/pull/5437)
- [**enhancement**][**ci**] Refine Simple CI error output [5435](https://github.com/Oneflow-Inc/oneflow/pull/5435)
- [**enhancement**][**ci**] Add conda env to Simple CI [5385](https://github.com/Oneflow-Inc/oneflow/pull/5385)
- [**enhancement**][**ci**] Fix clang plugin core file not found [5390](https://github.com/Oneflow-Inc/oneflow/pull/5390)
- [**bug**][**ci**] upload core when build with clang plugin [5384](https://github.com/Oneflow-Inc/oneflow/pull/5384)
- [**bug**][**ci**] clang plugin skip more files [5373](https://github.com/Oneflow-Inc/oneflow/pull/5373)
- [**enhancement**][**ci**] Use gh-action-scheduler-v2 [5370](https://github.com/Oneflow-Inc/oneflow/pull/5370)
- [**enhancement**][**ci**] relax speed threshold [5569](https://github.com/Oneflow-Inc/oneflow/pull/5569)
- [**bug**][**ci**] Fix wrong test path under compatible [5567](https://github.com/Oneflow-Inc/oneflow/pull/5567)
- [**enhancement**][**ci**][**need-simple-ci**] Prevent upload logs automatically [5560](https://github.com/Oneflow-Inc/oneflow/pull/5560)
- [**enhancement**][**ci**][**interface**] Add `nn.AdaptiveAvgPool1d` and `nn.AdaptiveAvgPool3d` [5445](https://github.com/Oneflow-Inc/oneflow/pull/5445)
- [**feature**][**ci**] add speed test in ci [5496](https://github.com/Oneflow-Inc/oneflow/pull/5496)
- [**enhancement**][**ci**] Reduce usage of Simple CI [5546](https://github.com/Oneflow-Inc/oneflow/pull/5546)
- [**feature**][**bug**][**ci**][**api**] Restruct upsample module [5524](https://github.com/Oneflow-Inc/oneflow/pull/5524)
- [**feature**][**ci**] multi client launcher test [5488](https://github.com/Oneflow-Inc/oneflow/pull/5488)
- [**enhancement**][**ci**] Remove automerge if cuda_new_interface failed [5519](https://github.com/Oneflow-Inc/oneflow/pull/5519)
- [**enhancement**][**ci**] Prevent adding subdir in python/test [5514](https://github.com/Oneflow-Inc/oneflow/pull/5514)
- [**enhancement**][**ci**] piprepo->pipindex [5517](https://github.com/Oneflow-Inc/oneflow/pull/5517)
- [**enhancement**][**ci**] add dynamic_loss_scale in ci tests [5337](https://github.com/Oneflow-Inc/oneflow/pull/5337)
- [**enhancement**][**ci**] Add timeout for wait_gpu_slot [5497](https://github.com/Oneflow-Inc/oneflow/pull/5497)
- [**enhancement**][**feature**][**ci**] new static check based on clang-tidy [5476](https://github.com/Oneflow-Inc/oneflow/pull/5476)
- [**enhancement**][**ci**] Fix url not downloadable in some browers [5701](https://github.com/Oneflow-Inc/oneflow/pull/5701)
- [**feature**][**ci**] multi client multi machine test [5685](https://github.com/Oneflow-Inc/oneflow/pull/5685)
- [**enhancement**][**ci**] Add cpu new interface CI [5639](https://github.com/Oneflow-Inc/oneflow/pull/5639)
- [**enhancement**][**ci**][**need-simple-ci**] Mv clangtidy to simple ci [5667](https://github.com/Oneflow-Inc/oneflow/pull/5667)
- [**enhancement**][**ci**][**need-simple-ci**] use clang tidy appimage in ci [5841](https://github.com/Oneflow-Inc/oneflow/pull/5841)
- [**enhancement**][**ci**] Use gcc 7 in release to prevent error [5840](https://github.com/Oneflow-Inc/oneflow/pull/5840)
- [**enhancement**][**ci**] bn tol 1e-4 => 1e-3 [5811](https://github.com/Oneflow-Inc/oneflow/pull/5811)
- [**enhancement**][**ci**] fix distributed run on built dir [5810](https://github.com/Oneflow-Inc/oneflow/pull/5810)
- [**enhancement**][**ci**] fix third party mirror check_sum [5802](https://github.com/Oneflow-Inc/oneflow/pull/5802)
- [**ci**][**documentation**] find more accurately which files need to be doctested [5782](https://github.com/Oneflow-Inc/oneflow/pull/5782)
- [**enhancement**][**ci**] Print stack unconditionally [5779](https://github.com/Oneflow-Inc/oneflow/pull/5779)
- [**enhancement**][**ci**][**need-simple-ci**] Enable more checkers for clang-tidy in CI [5738](https://github.com/Oneflow-Inc/oneflow/pull/5738)
- [**enhancement**][**ci**] CI: add clang-tidy check to test.yaml [5920](https://github.com/Oneflow-Inc/oneflow/pull/5920)
- [**ci**][**documentation**] fix docstring in oneflow.nn.functional namespace [5807](https://github.com/Oneflow-Inc/oneflow/pull/5807)
- [**enhancement**][**ci**] disable TREAT_WARNINGS_AS_ERRORS in Release CI [5886](https://github.com/Oneflow-Inc/oneflow/pull/5886)
- [**enhancement**][**ci**] Skip ci jobs by git diff [5863](https://github.com/Oneflow-Inc/oneflow/pull/5863)
- [**bug**][**ci**] quick fix 5978 [6030](https://github.com/Oneflow-Inc/oneflow/pull/6030)
- [**enhancement**][**bug**][**ci**] fix clang tidy diff options and file format [5990](https://github.com/Oneflow-Inc/oneflow/pull/5990)
- [**enhancement**][**ci**] add flow.relu [5847](https://github.com/Oneflow-Inc/oneflow/pull/5847)
- [**enhancement**][**ci**] equal => allclose [6164](https://github.com/Oneflow-Inc/oneflow/pull/6164)
- [**bug**][**ci**][**need-simple-ci**] CI: fix clang tidy checks in simple ci [6161](https://github.com/Oneflow-Inc/oneflow/pull/6161)
- [**enhancement**][**bug**][**ci**][**documentation**][**api**] add interpolate and layer_norm docs [6157](https://github.com/Oneflow-Inc/oneflow/pull/6157)
- [**bug**][**ci**] update speed test [6113](https://github.com/Oneflow-Inc/oneflow/pull/6113)
- [**enhancement**][**bug**][**ci**][**documentation**][**api**] speed import oneflow [6107](https://github.com/Oneflow-Inc/oneflow/pull/6107)
- [**bug**][**ci**] Also try install dev deps for CODEGEN_PYTHON_EXECUTABLE [6115](https://github.com/Oneflow-Inc/oneflow/pull/6115)
- [**bug**][**ci**][**need-simple-ci**] set gtest_CMAKE_DEBUG_POSTFIX "d" [6085](https://github.com/Oneflow-Inc/oneflow/pull/6085)
- [**enhancement**][**ci**] add cache init file for clang and CI build with clang [6062](https://github.com/Oneflow-Inc/oneflow/pull/6062)
- [**enhancement**][**ci**] add emoji in speed test output, make it continue-on-error [6214](https://github.com/Oneflow-Inc/oneflow/pull/6214)

Test enhancements:

- [**bug**][**test**][**interface**] Fix acos ci bug [5217](https://github.com/Oneflow-Inc/oneflow/pull/5217)
- [**feature**][**test**] implement automated test [5321](https://github.com/Oneflow-Inc/oneflow/pull/5321)
- [**enhancement**][**test**] move generator test into ops folder to accelerate tests [5472](https://github.com/Oneflow-Inc/oneflow/pull/5472)
- [**feature**][**test**][**api**] Add autotest part2 [5467](https://github.com/Oneflow-Inc/oneflow/pull/5467)
- [**enhancement**][**test**][**api**][**interface**] Add some tests with the new framework for auto testing [5561](https://github.com/Oneflow-Inc/oneflow/pull/5561)
- [**bug**][**test**] fix test error when do multi case test on graph [5590](https://github.com/Oneflow-Inc/oneflow/pull/5590)
- [**enhancement**][**test**] Refine module test using auto test by yaochi [5484](https://github.com/Oneflow-Inc/oneflow/pull/5484)
- [**enhancement**][**test**] Add autotest for BatchNorm2d [5734](https://github.com/Oneflow-Inc/oneflow/pull/5734)
- [**enhancement**][**test**] RTH_update_op_test [5823](https://github.com/Oneflow-Inc/oneflow/pull/5823)
- [**enhancement**][**test**] dev adamw graph config [5745](https://github.com/Oneflow-Inc/oneflow/pull/5745)
- [**feature**][**test**][**api**][**interface**] Add new autotest [5562](https://github.com/Oneflow-Inc/oneflow/pull/5562)
- [**bug**][**test**] restore test of alexnet graph [5798](https://github.com/Oneflow-Inc/oneflow/pull/5798)
- [**enhancement**][**test**][**interface**] add zhangshen op-test [5600](https://github.com/Oneflow-Inc/oneflow/pull/5600)
- [**feature**][**bug**][**tooling**][**test**][**interface**] Record autotest wrong code [5923](https://github.com/Oneflow-Inc/oneflow/pull/5923)
- [**enhancement**][**feature**][**test**][**api**] add randint [5718](https://github.com/Oneflow-Inc/oneflow/pull/5718)
- [**bug**][**test**] fix multi machine test [5984](https://github.com/Oneflow-Inc/oneflow/pull/5984)
- [**enhancement**][**test**][**interface**] some op test [6095](https://github.com/Oneflow-Inc/oneflow/pull/6095)

Tooling enhancements:

- [**bug**][**tooling**] user/summary: fix memory leak in `FillImageInSummary` [5742](https://github.com/Oneflow-Inc/oneflow/pull/5742)
- [**enhancement**][**tooling**][**cfg**] cfg: add move assignment operator for performance [5962](https://github.com/Oneflow-Inc/oneflow/pull/5962)
- [**enhancement**][**tooling**][**api**][**refactor**] refactor_all_device_placement_api [6080](https://github.com/Oneflow-Inc/oneflow/pull/6080)

Highlights

- First class support for eager execution. The deprecated APIs are moved to `oneflow.compatible.single_client`
- Drop-in replacement of `import torch` for existing Pytorch projects. You could test it by inter-changing `import oneflow as torch` and `import torch as flow`.
- [nn.Module](https://docs.oneflow.org/master/basics/04_build_network.html#module) for eager execution
- [nn.Graph](https://docs.oneflow.org/master/basics/08_nn_graph.html) for lazy execution
- [DDP](https://oneflow.readthedocs.io/en/master/nn.html#oneflow.nn.parallel.DistributedDataParallel) for data parallel

A sneak peek of the new API

Here is a minimum example showcasing how to incorporate a `nn.Module` in a `nn.Graph` and have it run in lazy mode.

python
class NeuralGraph(flow.nn.Graph):
def __init__(self, ...):
super().__init__()
self.model = model model is a nn.Module instance

def build(self, x):
y_pred = self.model(x)
return y_pred

graph = NeuralGraph() to create a nn.Graph instance
y_pred = graph(x) to run the created nn.Graph


New in Python API

- [**feature**][**eager**][**op**][**test**][**python**][**interface**] Add test for convtranspose2d [5239](https://github.com/Oneflow-Inc/oneflow/pull/5239)
- [**enhancement**][**python**][**interface**] Add GroupNorm [5175](https://github.com/Oneflow-Inc/oneflow/pull/5175)
- [**enhancement**][**eager**][**python**][**interface**] [Add] avgpool1d avgpool3d [5165](https://github.com/Oneflow-Inc/oneflow/pull/5165)
- [**feature**][**eager**][**op**][**python**][**interface**] Add deconv cpu impl [5224](https://github.com/Oneflow-Inc/oneflow/pull/5224)
- [**bug**][**eager**][**api**][**python**][**interface**] Fix acosh bug [5221](https://github.com/Oneflow-Inc/oneflow/pull/5221)
- [**feature**][**eager**][**op**][**python**][**interface**] Dev modules ctc loss [5168](https://github.com/Oneflow-Inc/oneflow/pull/5168)
- [**bottleneck**][**bug**][**documentation**][**python**][**interface**] Fix meshgrid test bug [5208](https://github.com/Oneflow-Inc/oneflow/pull/5208)
- [**eager**][**documentation**][**python**][**interface**] Rename CosineScheduler to CosineAnnealingLR [5112](https://github.com/Oneflow-Inc/oneflow/pull/5112)
- [**feature**][**eager**][**python**][**interface**] Add meshgrid module [5205](https://github.com/Oneflow-Inc/oneflow/pull/5205)
- [**enhancement**][**feature**][**bug**][**op**][**python**] support bias in conv2d's parameter list [5322](https://github.com/Oneflow-Inc/oneflow/pull/5322)
- [**eager**][**documentation**][**api**][**python**][**interface**] add not_equal, greater_equal and less_equal module [5350](https://github.com/Oneflow-Inc/oneflow/pull/5350)
- [**enhancement**][**eager**][**python**] refine pow module and its test [5319](https://github.com/Oneflow-Inc/oneflow/pull/5319)
- [**enhancement**][**eager**][**op**][**python**] Add triu op [5329](https://github.com/Oneflow-Inc/oneflow/pull/5329)
- [**enhancement**][**bug**][**python**] Fix optimizer for not supporting all kinds of iterables [5355](https://github.com/Oneflow-Inc/oneflow/pull/5355)
- [**bug**][**python**][**interface**] raise IndexError in get_canonical_index to support for loop [5345](https://github.com/Oneflow-Inc/oneflow/pull/5345)
- [**bug**][**python**][**interface**] tensor slice assign supports broadcasting [5344](https://github.com/Oneflow-Inc/oneflow/pull/5344)
- [**enhancement**][**op**][**python**] add cpu group conv logic [5314](https://github.com/Oneflow-Inc/oneflow/pull/5314)
- [**enhancement**][**python**] Add 'nn.Mish' module and corresponding functions [5310](https://github.com/Oneflow-Inc/oneflow/pull/5310)
- [**enhancement**][**build**][**python**] Remove ONNX from setup py [5297](https://github.com/Oneflow-Inc/oneflow/pull/5297)
- [**enhancement**][**python**][**interface**] [add] zeropad2d [5278](https://github.com/Oneflow-Inc/oneflow/pull/5278)
- [**feature**][**system**][**python**][**interface**] Lazy nn.Graph FeedInputOpExpr [5458](https://github.com/Oneflow-Inc/oneflow/pull/5458)
- [**feature**][**python**][**interface**] integrate nn.image.flip [5411](https://github.com/Oneflow-Inc/oneflow/pull/5411)
- [**bug**][**python**] Fix issues in point of MultiClientSession [5469](https://github.com/Oneflow-Inc/oneflow/pull/5469)
- [**enhancement**][**bug**][**python**] update HasAllMultiClientEnvVars() [5459](https://github.com/Oneflow-Inc/oneflow/pull/5459)
- [**enhancement**][**python**] Add in_top_k function [5428](https://github.com/Oneflow-Inc/oneflow/pull/5428)
- [**enhancement**][**python**] Dev add docstring [5449](https://github.com/Oneflow-Inc/oneflow/pull/5449)
- [**feature**][**api**][**python**] MultiClientSession [5407](https://github.com/Oneflow-Inc/oneflow/pull/5407)
- [**documentation**][**python**] remove --user [5431](https://github.com/Oneflow-Inc/oneflow/pull/5431)
- [**feature**][**python**][**interface**] nn.Graph python [5309](https://github.com/Oneflow-Inc/oneflow/pull/5309)
- [**feature**][**python**][**interface**] Fea/nn graph/graph name [5413](https://github.com/Oneflow-Inc/oneflow/pull/5413)
- [**bug**][**python**][**interface**] rm nn.Graph.train [5424](https://github.com/Oneflow-Inc/oneflow/pull/5424)
- [**op**][**documentation**][**api**][**python**][**interface**] add bernoulli module [5353](https://github.com/Oneflow-Inc/oneflow/pull/5353)
- [**enhancement**][**python**] flow.S/B/P [5306](https://github.com/Oneflow-Inc/oneflow/pull/5306)
- [**enhancement**][**documentation**][**python**] Add instruction on upgrade pip [5400](https://github.com/Oneflow-Inc/oneflow/pull/5400)
- [**enhancement**][**python**] Rm oneflow export and experimental [5589](https://github.com/Oneflow-Inc/oneflow/pull/5589)
- [**bug**][**python**] Fix nn.graph.utils module conflict [5598](https://github.com/Oneflow-Inc/oneflow/pull/5598)
- [**feature**][**ci**][**python**] Update autotest framework [5520](https://github.com/Oneflow-Inc/oneflow/pull/5520)
- [**enhancement**][**python**] copy of_proto_python_dir to compatible_single_client_python [5539](https://github.com/Oneflow-Inc/oneflow/pull/5539)
- [**enhancement**][**api**][**python**] del default env init [5537](https://github.com/Oneflow-Inc/oneflow/pull/5537)
- [**enhancement**][**python**] Fix single client using same glog file [5535](https://github.com/Oneflow-Inc/oneflow/pull/5535)
- [**bug**][**api**][**python**] Fix Session TryClose [5531](https://github.com/Oneflow-Inc/oneflow/pull/5531)
- [**enhancement**][**feature**][**python**] split vector-matrix norm [5478](https://github.com/Oneflow-Inc/oneflow/pull/5478)
- [**feature**][**eager**][**op**][**python**][**interface**] Add more upsample kernel [5382](https://github.com/Oneflow-Inc/oneflow/pull/5382)
- [**enhancement**][**feature**][**test**][**python**] add torchstyle unittest [5489](https://github.com/Oneflow-Inc/oneflow/pull/5489)
- [**feature**][**system**][**python**] nn.Graph with training [5662](https://github.com/Oneflow-Inc/oneflow/pull/5662)
- [**enhancement**][**feature**][**python**] Fea/nn graph/block proxy func [5727](https://github.com/Oneflow-Inc/oneflow/pull/5727)
- [**enhancement**][**api**][**python**] consistent_tensor_to_api [5703](https://github.com/Oneflow-Inc/oneflow/pull/5703)
- [**feature**][**eager**][**op**][**python**] Dev Align torch avgpool [5610](https://github.com/Oneflow-Inc/oneflow/pull/5610)
- [**enhancement**][**python**] fix circular deps of sbp python module [5706](https://github.com/Oneflow-Inc/oneflow/pull/5706)
- [**documentation**][**python**] [part5]Remove singleclient outdated api [5674](https://github.com/Oneflow-Inc/oneflow/pull/5674)
- [**enhancement**][**python**] [part4]Remove singleclient outdated api [5672](https://github.com/Oneflow-Inc/oneflow/pull/5672)
- [**bug**][**op**][**python**] remove outdated code in conv3d [5696](https://github.com/Oneflow-Inc/oneflow/pull/5696)
- [**enhancement**][**test**][**python**] enlarge tolerance of dataloader test [5689](https://github.com/Oneflow-Inc/oneflow/pull/5689)
- [**enhancement**][**test**][**python**] add autotest for some math ops [5646](https://github.com/Oneflow-Inc/oneflow/pull/5646)
- [**feature**][**python**] nn.Graph optimizer part 2: add L2, pass job complete, refactor [5604](https://github.com/Oneflow-Inc/oneflow/pull/5604)
- [**enhancement**][**python**] Add clip_grad_norm [5299](https://github.com/Oneflow-Inc/oneflow/pull/5299)
- [**purge**][**python**] Remove Single-Client API in oneflow default python [5827](https://github.com/Oneflow-Inc/oneflow/pull/5827)
- [**bug**][**python**] Fix ddp grad size [5834](https://github.com/Oneflow-Inc/oneflow/pull/5834)
- [**enhancement**][**feature**][**python**] Dev RMSprop graph conf [5768](https://github.com/Oneflow-Inc/oneflow/pull/5768)
- [**enhancement**][**purge**][**eager**][**python**] remove scale arg in optimizer [5821](https://github.com/Oneflow-Inc/oneflow/pull/5821)
- [**enhancement**][**feature**][**python**] graph/block io check [5803](https://github.com/Oneflow-Inc/oneflow/pull/5803)
- [**enhancement**][**feature**][**python**] Dev adam graph conf [5709](https://github.com/Oneflow-Inc/oneflow/pull/5709)
- [**purge**][**python**] [part10]Remove singleclient outdated api [5756](https://github.com/Oneflow-Inc/oneflow/pull/5756)
- [**feature**][**api**][**python**] better repr of nn.Graph for debug [5762](https://github.com/Oneflow-Inc/oneflow/pull/5762)
- [**bug**][**python**] fix weight decay in RMSprop [5755](https://github.com/Oneflow-Inc/oneflow/pull/5755)
- [**purge**][**python**] [part9]Remove singleclient outdated api [5752](https://github.com/Oneflow-Inc/oneflow/pull/5752)
- [**purge**][**python**] [part8]Remove singleclient outdated api [5750](https://github.com/Oneflow-Inc/oneflow/pull/5750)
- [**documentation**][**python**] add first batch of methods in oneflow.nn.functional namespace [5693](https://github.com/Oneflow-Inc/oneflow/pull/5693)
- [**purge**][**python**] [part6]Remove singleclient outdated api [5704](https://github.com/Oneflow-Inc/oneflow/pull/5704)
- [**bug**][**python**] use default_generator.seed() as random_seed in init [5721](https://github.com/Oneflow-Inc/oneflow/pull/5721)
- [**bug**][**system**][**python**] ddp broadcast params and buffers [5913](https://github.com/Oneflow-Inc/oneflow/pull/5913)
- [**enhancement**][**test**][**python**] Add consistent tensor requires grad test [5925](https://github.com/Oneflow-Inc/oneflow/pull/5925)
- [**bug**][**python**] wrap flow.nn.init.\* with flow.no_grad() [5932](https://github.com/Oneflow-Inc/oneflow/pull/5932)
- [**feature**][**api**][**python**][**interface**] add clip_grad to optimizer [5817](https://github.com/Oneflow-Inc/oneflow/pull/5817)
- [**enhancement**][**ci**][**op**][**test**][**python**] add randperm with test and docs [5680](https://github.com/Oneflow-Inc/oneflow/pull/5680)
- [**feature**][**api**][**python**] Fea/nn graph/ lr_schedule(and cosine lr_sch) and opt_group [5846](https://github.com/Oneflow-Inc/oneflow/pull/5846)
- [**bug**][**python**] fix bug of SyncOnMasterFn atexit [5909](https://github.com/Oneflow-Inc/oneflow/pull/5909)
- [**purge**][**python**] Delete single client nn modules [6061](https://github.com/Oneflow-Inc/oneflow/pull/6061)
- [**enhancement**][**python**] Move framework.distribute to env [6022](https://github.com/Oneflow-Inc/oneflow/pull/6022)
- [**bug**][**python**] skip sync when abnormally exiting [6025](https://github.com/Oneflow-Inc/oneflow/pull/6025)
- [**feature**][**python**] Fea/nn graph/warmup amp config [5969](https://github.com/Oneflow-Inc/oneflow/pull/5969)
- [**documentation**][**python**] add optimizer api docs [6131](https://github.com/Oneflow-Inc/oneflow/pull/6131)
- [**documentation**][**python**] add_tensor_api_doc [6127](https://github.com/Oneflow-Inc/oneflow/pull/6127)
- [**bug**][**python**] Fix test_grid_sample.py and test_affine_grid.py threshold [6125](https://github.com/Oneflow-Inc/oneflow/pull/6125)
- [**documentation**][**api**][**python**] add doc of graph [6093](https://github.com/Oneflow-Inc/oneflow/pull/6093)
- [**bug**][**python**] Fix make of_format fail in ubuntu [6120](https://github.com/Oneflow-Inc/oneflow/pull/6120)
- [**feature**][**api**][**python**][**interface**] Fea/graph helpers [6088](https://github.com/Oneflow-Inc/oneflow/pull/6088)
- [**enhancement**][**eager**][**python**][**interface**] Use flow.randint in dataloader [6086](https://github.com/Oneflow-Inc/oneflow/pull/6086)
- [**feature**][**eager**][**api**][**python**][**interface**] Import oneflow as torch [6076](https://github.com/Oneflow-Inc/oneflow/pull/6076)
- [**enhancement**][**test**][**api**][**python**][**refactor**] rename OfrecordReader to OFRcordReader [6090](https://github.com/Oneflow-Inc/oneflow/pull/6090)
- [**purge**][**python**][**need-single-client-tests**] Delete single client nn modules [6082](https://github.com/Oneflow-Inc/oneflow/pull/6082)
- [**enhancement**][**python**] flow.load tolerates FileNotFound fault [6083](https://github.com/Oneflow-Inc/oneflow/pull/6083)
- [**feature**][**python**] Fea/pipeline in graph [6105](https://github.com/Oneflow-Inc/oneflow/pull/6105)
- [**enhancement**][**test**][**python**] graph activation checkpointing [6192](https://github.com/Oneflow-Inc/oneflow/pull/6192)
- [**enhancement**][**feature**][**op**][**python**] rnn test [6165](https://github.com/Oneflow-Inc/oneflow/pull/6165)

New in Ops:

- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part2: Add partial unary and math functional apis [5218](https://github.com/Oneflow-Inc/oneflow/pull/5218)
- [**enhancement**][**bug**][**op**][**interface**] Refine deconv kernel [5229](https://github.com/Oneflow-Inc/oneflow/pull/5229)
- [**enhancement**][**op**][**api**][**interface**] add ReflectionPad2d [5172](https://github.com/Oneflow-Inc/oneflow/pull/5172)
- [**feature**][**eager**][**op**][**api**][**interface**] crossentropyloss and nllloss support ignore_index [5195](https://github.com/Oneflow-Inc/oneflow/pull/5195)
- [**feature**][**eager**][**op**][**api**][**interface**] Yejiaojiao/dev bcewithlogitsloss [5173](https://github.com/Oneflow-Inc/oneflow/pull/5173)
- [**bug**][**ci**][**op**] Dev user op set default is_dynamic [5223](https://github.com/Oneflow-Inc/oneflow/pull/5223)
- [**enhancement**][**op**] add magic method for pow [5199](https://github.com/Oneflow-Inc/oneflow/pull/5199)
- [**enhancement**][**op**][**interface**] add cpu version of upsampling [5194](https://github.com/Oneflow-Inc/oneflow/pull/5194)
- [**enhancement**][**bug**][**op**][**api**][**interface**] add ReplicationPad2d [5148](https://github.com/Oneflow-Inc/oneflow/pull/5148)
- [**feature**][**eager**][**op**][**api**][**interface**] add kldivloss module [5155](https://github.com/Oneflow-Inc/oneflow/pull/5155)
- [**feature**][**eager**][**op**][**documentation**][**build**][**api**][**interface**] Add floor module and the corresponding testcases [4964](https://github.com/Oneflow-Inc/oneflow/pull/4964)
- [**enhancement**][**feature**][**op**] Dev conv1d module [5280](https://github.com/Oneflow-Inc/oneflow/pull/5280)
- [**enhancement**][**op**] Add ctc_greedy_decoder op [5294](https://github.com/Oneflow-Inc/oneflow/pull/5294)
- [**enhancement**][**op**][**system**] Dev remove default grad func [5320](https://github.com/Oneflow-Inc/oneflow/pull/5320)
- [**enhancement**][**op**][**system**] Add pad grad func. [5354](https://github.com/Oneflow-Inc/oneflow/pull/5354)
- [**enhancement**][**op**][**system**] Add gradient funcs. [5348](https://github.com/Oneflow-Inc/oneflow/pull/5348)
- [**feature**][**purge**][**bug**][**eager**][**op**][**interface**] fix upsample nearest bug [5347](https://github.com/Oneflow-Inc/oneflow/pull/5347)
- [**enhancement**][**op**][**system**] [Functional] Part7: Migrate pooling ops [5253](https://github.com/Oneflow-Inc/oneflow/pull/5253)
- [**enhancement**][**op**] nvjpeg hardware acc [5240](https://github.com/Oneflow-Inc/oneflow/pull/5240)
- [**enhancement**][**feature**][**ci**][**eager**][**op**][**api**][**interface**] Add bmm module [5334](https://github.com/Oneflow-Inc/oneflow/pull/5334)
- [**enhancement**][**eager**][**op**] Dev image decode eager [5333](https://github.com/Oneflow-Inc/oneflow/pull/5333)
- [**enhancement**][**op**] Optimize softmax warp impl [4977](https://github.com/Oneflow-Inc/oneflow/pull/4977)
- [**enhancement**][**eager**][**op**] Dev tensor buffer eager [5317](https://github.com/Oneflow-Inc/oneflow/pull/5317)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part6: Migrate conv op [5252](https://github.com/Oneflow-Inc/oneflow/pull/5252)
- [**enhancement**][**eager**][**op**] Dev sort eager [5284](https://github.com/Oneflow-Inc/oneflow/pull/5284)
- [**enhancement**][**bug**][**op**][**api**] fix bceloss bug in default weight and reduction [5303](https://github.com/Oneflow-Inc/oneflow/pull/5303)
- [**bug**][**eager**][**op**] remove redundant assert and check [5264](https://github.com/Oneflow-Inc/oneflow/pull/5264)
- [**enhancement**][**bug**][**ci**][**op**] fix bceloss bug about weight [5269](https://github.com/Oneflow-Inc/oneflow/pull/5269)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part5: Migrate nn ops [5249](https://github.com/Oneflow-Inc/oneflow/pull/5249)
- [**enhancement**][**eager**][**op**] Dev argsort eager [5273](https://github.com/Oneflow-Inc/oneflow/pull/5273)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part4: Migrate array ops [5247](https://github.com/Oneflow-Inc/oneflow/pull/5247)
- [**enhancement**][**op**][**api**][**refactor**] [Functional] Part3: Migrate binary and activation ops [5246](https://github.com/Oneflow-Inc/oneflow/pull/5246)
- [**bug**][**ci**][**op**][**test**] Dev fix rmsprop ci fail [5481](https://github.com/Oneflow-Inc/oneflow/pull/5481)
- [**enhancement**][**op**] add inplace method: Tensor.sin\_ [5471](https://github.com/Oneflow-Inc/oneflow/pull/5471)
- [**bug**][**op**] hotfix image_batch_align [5461](https://github.com/Oneflow-Inc/oneflow/pull/5461)
- [**enhancement**][**eager**][**op**][**interface**] Dev maxpool series op 123d [5244](https://github.com/Oneflow-Inc/oneflow/pull/5244)
- [**bug**][**op**] fix pool gpu kernel [5446](https://github.com/Oneflow-Inc/oneflow/pull/5446)
- [**feature**][**eager**][**op**][**api**][**interface**] add pixelshufflev2 module [5383](https://github.com/Oneflow-Inc/oneflow/pull/5383)
- [**enhancement**][**feature**][**ci**][**eager**][**op**][**documentation**][**api**][**interface**] Add flow xxx and tensor xxx autotest [5386](https://github.com/Oneflow-Inc/oneflow/pull/5386)
- [**enhancement**][**feature**][**eager**][**op**][**api**][**interface**] Modules chunk [5324](https://github.com/Oneflow-Inc/oneflow/pull/5324)
- [**enhancement**][**eager**][**op**] add image normalize for eager [5402](https://github.com/Oneflow-Inc/oneflow/pull/5402)
- [**enhancement**][**eager**][**op**] Dev batch align module [5401](https://github.com/Oneflow-Inc/oneflow/pull/5401)
- [**enhancement**][**eager**][**op**] add coco reader module [5391](https://github.com/Oneflow-Inc/oneflow/pull/5391)
- [**enhancement**][**wip**][**op**] Restruct Elementwise kernel [4130](https://github.com/Oneflow-Inc/oneflow/pull/4130)
- [**bug**][**op**] Fix DecodeRandom reuse mem [5606](https://github.com/Oneflow-Inc/oneflow/pull/5606)
- [**enhancement**][**op**] Align pytorch maxpool [5525](https://github.com/Oneflow-Inc/oneflow/pull/5525)
- [**enhancement**][**bottleneck**][**eager**][**op**][**api**] implementation of constantpad-3d op [5529](https://github.com/Oneflow-Inc/oneflow/pull/5529)
- [**enhancement**][**eager**][**op**] Add scale size for resize [5509](https://github.com/Oneflow-Inc/oneflow/pull/5509)
- [**enhancement**][**op**][**api**][**refactor**] Dev optimize tensor setitem [5501](https://github.com/Oneflow-Inc/oneflow/pull/5501)
- [**enhancement**][**op**] register uint8 dtypeto support dataloader [5499](https://github.com/Oneflow-Inc/oneflow/pull/5499)
- [**enhancement**][**op**] Add unique.cuh [5487](https://github.com/Oneflow-Inc/oneflow/pull/5487)
- [**enhancement**][**op**][**api**][**interface**] Dev ofrecord auto truncating [5412](https://github.com/Oneflow-Inc/oneflow/pull/5412)
- [**feature**][**op**][**system**][**interface**] Feat: LazyInterpret::ApplyImpl support SourceUserOpExpr and Copy [5711](https://github.com/Oneflow-Inc/oneflow/pull/5711)
- [**enhancement**][**op**][**interface**] Dev logical_and/or modules [5636](https://github.com/Oneflow-Inc/oneflow/pull/5636)
- [**enhancement**][**op**] support any number positional arguments for ones and zeros op [5698](https://github.com/Oneflow-Inc/oneflow/pull/5698)
- [**enhancement**][**feature**][**eager**][**op**] Add conv3d Module [5327](https://github.com/Oneflow-Inc/oneflow/pull/5327)
- [**feature**][**eager**][**op**][**api**][**interface**] add batchnorm3d module [5631](https://github.com/Oneflow-Inc/oneflow/pull/5631)
- [**bug**][**eager**][**op**] fix reduce min max backward bug [5651](https://github.com/Oneflow-Inc/oneflow/pull/5651)
- [**enhancement**][**op**] Debug dim scatter [5371](https://github.com/Oneflow-Inc/oneflow/pull/5371)
- [**enhancement**][**op**][**interface**] Dev eye [5583](https://github.com/Oneflow-Inc/oneflow/pull/5583)
- [**enhancement**][**eager**][**op**] Dev minimum maximum [5576](https://github.com/Oneflow-Inc/oneflow/pull/5576)
- [**enhancement**][**op**] Restruct activation grad op [5669](https://github.com/Oneflow-Inc/oneflow/pull/5669)
- [**enhancement**][**feature**][**eager**][**op**] Rewrite activation function [5465](https://github.com/Oneflow-Inc/oneflow/pull/5465)
- [**bug**][**op**][**documentation**] add oneflow.cat for documentation [5621](https://github.com/Oneflow-Inc/oneflow/pull/5621)
- [**enhancement**][**op**] Lcy logsoftmax [5746](https://github.com/Oneflow-Inc/oneflow/pull/5746)
- [**feature**][**op**][**need-simple-ci**] Feat empty op [5659](https://github.com/Oneflow-Inc/oneflow/pull/5659)
- [**enhancement**][**eager**][**op**] Dev split [5714](https://github.com/Oneflow-Inc/oneflow/pull/5714)
- [**enhancement**][**op**][**interface**] add index_select op [5661](https://github.com/Oneflow-Inc/oneflow/pull/5661)
- [**bug**][**op**] fix nvjpeg hw acc [5851](https://github.com/Oneflow-Inc/oneflow/pull/5851)
- [**enhancement**][**op**] Remove move in conv_cudnn [5828](https://github.com/Oneflow-Inc/oneflow/pull/5828)
- [**enhancement**][**op**][**interface**] Dev logical_xor module [5694](https://github.com/Oneflow-Inc/oneflow/pull/5694)
- [**bug**][**eager**][**op**] fix squeeze [5808](https://github.com/Oneflow-Inc/oneflow/pull/5808)
- [**enhancement**][**op**] Get parallel_id and parallel_num through rank and world size in DDP [5717](https://github.com/Oneflow-Inc/oneflow/pull/5717)
- [**bug**][**eager**][**op**] delete interpolate int type [5805](https://github.com/Oneflow-Inc/oneflow/pull/5805)
- [**bug**][**op**] Fix bug in scatter [5743](https://github.com/Oneflow-Inc/oneflow/pull/5743)
- [**enhancement**][**op**] Refactor: remove module not required, call function directly [5754](https://github.com/Oneflow-Inc/oneflow/pull/5754)
- [**enhancement**][**op**] Remove modules not required(tan, erfc, log1p, scatter_nd) [5791](https://github.com/Oneflow-Inc/oneflow/pull/5791)
- [**enhancement**][**op**] Refactor scatter, clamp and pow in cpp instead of in python [5715](https://github.com/Oneflow-Inc/oneflow/pull/5715)
- [**enhancement**][**op**] Rm useless code in gather files [5687](https://github.com/Oneflow-Inc/oneflow/pull/5687)
- [**enhancement**][**eager**][**op**] change flip_code to scalar [5786](https://github.com/Oneflow-Inc/oneflow/pull/5786)
- [**enhancement**][**bug**][**op**][**interface**] fix upsample bug [5753](https://github.com/Oneflow-Inc/oneflow/pull/5753)
- [**bug**][**op**][**interface**] Quick fix Lazy nn.Graph input/output OpConf.BlobConf.is_dynamic [5767](https://github.com/Oneflow-Inc/oneflow/pull/5767)
- [**enhancement**][**bug**][**eager**][**op**] fix argwhere 0-dim bug [5760](https://github.com/Oneflow-Inc/oneflow/pull/5760)
- [**enhancement**][**eager**][**op**] delete unused code [5744](https://github.com/Oneflow-Inc/oneflow/pull/5744)
- [**feature**][**op**] Export fused_scale_tril op [5933](https://github.com/Oneflow-Inc/oneflow/pull/5933)
- [**bug**][**op**] Fix backward bug in 3d [5908](https://github.com/Oneflow-Inc/oneflow/pull/5908)
- [**bug**][**op**] Fix one_hot api limit [5927](https://github.com/Oneflow-Inc/oneflow/pull/5927)
- [**enhancement**][**eager**][**op**] Dev where scalar [5797](https://github.com/Oneflow-Inc/oneflow/pull/5797)
- [**bug**][**op**] fix grad error [5914](https://github.com/Oneflow-Inc/oneflow/pull/5914)
- [**feature**][**bug**][**op**] Fix inplace op circle reference bug [5910](https://github.com/Oneflow-Inc/oneflow/pull/5910)
- [**enhancement**][**op**] Move the judgment content to c++， And add scalar fmod [5854](https://github.com/Oneflow-Inc/oneflow/pull/5854)
- [**enhancement**][**op**] Support combined_margin_loss op in flow.nn.modules [5830](https://github.com/Oneflow-Inc/oneflow/pull/5830)
- [**enhancement**][**op**][**api**][**interface**] functional_one_hot [5315](https://github.com/Oneflow-Inc/oneflow/pull/5315)
- [**enhancement**][**op**] Dev scalar op [5778](https://github.com/Oneflow-Inc/oneflow/pull/5778)
- [**bug**][**eager**][**op**] fix gather kernel 0 shape [5888](https://github.com/Oneflow-Inc/oneflow/pull/5888)
- [**enhancement**][**op**] add l2_normalize for mutl-client interfaces [5859](https://github.com/Oneflow-Inc/oneflow/pull/5859)
- [**feature**][**op**] Export function softmax_cross_entropy [6056](https://github.com/Oneflow-Inc/oneflow/pull/6056)
- [**enhancement**][**op**] Add int attr for functional adaptive average pool [6059](https://github.com/Oneflow-Inc/oneflow/pull/6059)
- [**enhancement**][**op**][**interface**] dev full op [5955](https://github.com/Oneflow-Inc/oneflow/pull/5955)
- [**bug**][**eager**][**op**] fix 0dim inplace add [6029](https://github.com/Oneflow-Inc/oneflow/pull/6029)
- [**feature**][**op**][**system**][**interface**] Feat: nn.Graph image gpu decoder [6014](https://github.com/Oneflow-Inc/oneflow/pull/6014)
- [**enhancement**][**op**][**interface**] dev optim_optim_lr_scheduler_multisteplr [5975](https://github.com/Oneflow-Inc/oneflow/pull/5975)
- [**enhancement**][**op**] NopKernel [6035](https://github.com/Oneflow-Inc/oneflow/pull/6035)
- [**enhancement**][**eager**][**op**][**api**] Dev tril op [6005](https://github.com/Oneflow-Inc/oneflow/pull/6005)
- [**enhancement**][**op**] dev unfold and fold [5675](https://github.com/Oneflow-Inc/oneflow/pull/5675)
- [**enhancement**][**op**] ResNet CUDA Graphs [6018](https://github.com/Oneflow-Inc/oneflow/pull/6018)
- [**enhancement**][**feature**][**op**] add broadcast pow [6013](https://github.com/Oneflow-Inc/oneflow/pull/6013)
- [**enhancement**][**op**][**interface**] init of op diag [5298](https://github.com/Oneflow-Inc/oneflow/pull/5298)
- [**op**][**documentation**][**api**] Fix api document bug [6009](https://github.com/Oneflow-Inc/oneflow/pull/6009)
- [**enhancement**][**op**] Dev fused functional [5954](https://github.com/Oneflow-Inc/oneflow/pull/5954)
- [**bug**][**op**][**build**] Add nvcc flag -Werror cross-execution-space-call [6002](https://github.com/Oneflow-Inc/oneflow/pull/6002)
- [**bug**][**op**] Fix Normalization grad function [5993](https://github.com/Oneflow-Inc/oneflow/pull/5993)
- [**enhancement**][**feature**][**eager**][**op**][**test**][**interface**] Add fused self attention [5966](https://github.com/Oneflow-Inc/oneflow/pull/5966)
- [**enhancement**][**bug**][**ci**][**eager**][**op**][**api**][**interface**] Try to fix var bug [5973](https://github.com/Oneflow-Inc/oneflow/pull/5973)
- [**enhancement**][**feature**][**eager**][**op**][**interface**] add prod op [5867](https://github.com/Oneflow-Inc/oneflow/pull/5867)
- [**enhancement**][**eager**][**op**][**api**] add glu op [6065](https://github.com/Oneflow-Inc/oneflow/pull/6065)
- [**enhancement**][**op**] Align Torch.nn.functional poolXd [6184](https://github.com/Oneflow-Inc/oneflow/pull/6184)
- [**bug**][**eager**][**op**] fix backward index for gamma beta [6149](https://github.com/Oneflow-Inc/oneflow/pull/6149)
- [**bug**][**op**][**system**] Fix BroadcastMatmulGrad bug [6168](https://github.com/Oneflow-Inc/oneflow/pull/6168)
- [**enhancement**][**op**][**api**] Add Int support for functional.avg/maxpool [6174](https://github.com/Oneflow-Inc/oneflow/pull/6174)
- [**bug**][**eager**][**op**][**api**][**interface**] align dropout api name with pytorch [6170](https://github.com/Oneflow-Inc/oneflow/pull/6170)
- [**enhancement**][**op**] support inplace operation for hardsigmoid [6137](https://github.com/Oneflow-Inc/oneflow/pull/6137)
- [**enhancement**][**bug**][**op**] Fix do bias correction in Adam/AdamW [5960](https://github.com/Oneflow-Inc/oneflow/pull/5960)
- [**bug**][**eager**][**op**][**api**][**interface**] fix repeat 0-dim tensor bug [6150](https://github.com/Oneflow-Inc/oneflow/pull/6150)
- [**enhancement**][**bug**][**op**] Fix select_first_grad bug [6142](https://github.com/Oneflow-Inc/oneflow/pull/6142)
- [**bug**][**ci**][**eager**][**op**][**documentation**][**interface**] Add clipgrad doc and contiguous [6130](https://github.com/Oneflow-Inc/oneflow/pull/6130)
- [**bug**][**op**] Fix eager optim dynamic attr bug [6111](https://github.com/Oneflow-Inc/oneflow/pull/6111)
- [**enhancement**][**op**] Support grid_sample and affine_grid operator [6038](https://github.com/Oneflow-Inc/oneflow/pull/6038)
- [**op**][**documentation**] Export apis for documentation [6068](https://github.com/Oneflow-Inc/oneflow/pull/6068)
- [**enhancement**][**feature**][**bug**][**ci**][**eager**][**op**][**documentation**][**interface**] transfer python function to c++ method [6114](https://github.com/Oneflow-Inc/oneflow/pull/6114)
- [**op**][**documentation**] Dev functional batch_gather [6233](https://github.com/Oneflow-Inc/oneflow/pull/6233)
- [**enhancement**][**op**][**test**] fix cross_entropy_loss and its test [5799](https://github.com/Oneflow-Inc/oneflow/pull/5799)
- [**bug**][**op**] Use attr nd_sbp to check consistent [6222](https://github.com/Oneflow-Inc/oneflow/pull/6222)
- [**enhancement**][**op**] Dev fused bn functional [6077](https://github.com/Oneflow-Inc/oneflow/pull/6077)
- [**enhancement**][**op**] support default value in intlist [6201](https://github.com/Oneflow-Inc/oneflow/pull/6201)
- [**bug**][**op**] fix sparse_softmax get_nd_sbp [6203](https://github.com/Oneflow-Inc/oneflow/pull/6203)
- [**bug**][**op**] Fix bug in model fused update [6197](https://github.com/Oneflow-Inc/oneflow/pull/6197)
- [**enhancement**][**op**][**system**][**refactor**] Optimize tensor getitem. [5433](https://github.com/Oneflow-Inc/oneflow/pull/5433)

New in Eager:

- [**enhancement**][**eager**][**interface**] Reconstruct module files [5251](https://github.com/Oneflow-Inc/oneflow/pull/5251)
- [**bug**][**eager**][**documentation**][**interface**] Fix conv module bug [5245](https://github.com/Oneflow-Inc/oneflow/pull/5245)
- [**bug**][**ci**][**eager**][**interface**] Fix bce withlogitloss ci error [5237](https://github.com/Oneflow-Inc/oneflow/pull/5237)
- [**feature**][**eager**][**api**][**interface**] module BCELoss [5144](https://github.com/Oneflow-Inc/oneflow/pull/5144)
- [**enhancement**][**feature**][**eager**][**api**][**interface**] Dev norm op [5178](https://github.com/Oneflow-Inc/oneflow/pull/5178)
- [**enhancement**][**bug**][**eager**] Fix stack module [5222](https://github.com/Oneflow-Inc/oneflow/pull/5222)
- [**enhancement**][**feature**][**eager**] Support different dtype of equal module [5214](https://github.com/Oneflow-Inc/oneflow/pull/5214)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**][**interface**] Add nllloss backward [5210](https://github.com/Oneflow-Inc/oneflow/pull/5210)
- [**enhancement**][**eager**][**api**][**upload-core**] Decouple FileSystem and IOConf [5162](https://github.com/Oneflow-Inc/oneflow/pull/5162)
- [**enhancement**][**ci**][**eager**] Set lower precision avoid ci failing [5200](https://github.com/Oneflow-Inc/oneflow/pull/5200)
- [**eager**][**documentation**] Add hint when apply FunctionNode second time [5369](https://github.com/Oneflow-Inc/oneflow/pull/5369)
- [**enhancement**][**feature**][**bug**][**ci**][**eager**][**documentation**][**api**] Fix upsample bilinear bug [5366](https://github.com/Oneflow-Inc/oneflow/pull/5366)
- [**bug**][**eager**] Fix not contiguous ndarray to tensor bug [5351](https://github.com/Oneflow-Inc/oneflow/pull/5351)
- [**enhancement**][**eager**][**system**] Infer consistent tensor meta [5118](https://github.com/Oneflow-Inc/oneflow/pull/5118)
- [**feature**][**eager**] Feat graph autograd engine [5296](https://github.com/Oneflow-Inc/oneflow/pull/5296)
- [**enhancement**][**eager**][**interface**] Dev type as module [5349](https://github.com/Oneflow-Inc/oneflow/pull/5349)
- [**feature**][**eager**][**documentation**][**api**][**interface**] Add new ones module [5342](https://github.com/Oneflow-Inc/oneflow/pull/5342)
- [**enhancement**][**bug**][**eager**] Fix logical slice assign dtype [5339](https://github.com/Oneflow-Inc/oneflow/pull/5339)
- [**bug**][**ci**][**eager**][**documentation**][**api**][**interface**] Fix where module bug [5300](https://github.com/Oneflow-Inc/oneflow/pull/5300)
- [**bug**][**ci**][**eager**][**documentation**][**api**] Fix l1loss ci error [5307](https://github.com/Oneflow-Inc/oneflow/pull/5307)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**][**interface**] Qi's First Edit of deleting "print" and ".numpy" [5129](https://github.com/Oneflow-Inc/oneflow/pull/5129)
- [**feature**][**eager**][**refactor**] Separate autograd meta to tensor [5267](https://github.com/Oneflow-Inc/oneflow/pull/5267)
- [**feature**][**eager**][**api**][**interface**] add tile module [5234](https://github.com/Oneflow-Inc/oneflow/pull/5234)
- [**enhancement**][**eager**] Release lambda function to reuse tensor memory [5266](https://github.com/Oneflow-Inc/oneflow/pull/5266)
- [**feature**][**bug**][**eager**][**documentation**] Fix default value not set bug [5483](https://github.com/Oneflow-Inc/oneflow/pull/5483)
- [**enhancement**][**eager**][**interface**] [Add] gather_nd scatter_nd [5422](https://github.com/Oneflow-Inc/oneflow/pull/5422)
- [**enhancement**][**bug**][**eager**] fix param [5473](https://github.com/Oneflow-Inc/oneflow/pull/5473)
- [**bug**][**eager**] Fix Tensor.grad setter bug [5462](https://github.com/Oneflow-Inc/oneflow/pull/5462)
- [**enhancement**][**eager**] Rename now_grad_arg to current_grad [5466](https://github.com/Oneflow-Inc/oneflow/pull/5466)
- [**eager**][**test**][**documentation**][**interface**] Add autotest part1 [5436](https://github.com/Oneflow-Inc/oneflow/pull/5436)
- [**enhancement**][**eager**] Use functional copy instead of op_builder [5460](https://github.com/Oneflow-Inc/oneflow/pull/5460)
- [**bottleneck**][**bug**][**eager**][**interface**] fix -1 index not support bug [5448](https://github.com/Oneflow-Inc/oneflow/pull/5448)
- [**bug**][**ci**][**eager**][**documentation**][**api**] Fix concat backward bug [5443](https://github.com/Oneflow-Inc/oneflow/pull/5443)
- [**enhancement**][**bug**][**ci**][**eager**] Add autograd engine warning [5444](https://github.com/Oneflow-Inc/oneflow/pull/5444)
- [**feature**][**eager**][**api**][**interface**] Smoothl1loss [5256](https://github.com/Oneflow-Inc/oneflow/pull/5256)
- [**enhancement**][**bottleneck**][**eager**] remove device dtype params [5434](https://github.com/Oneflow-Inc/oneflow/pull/5434)
- [**bug**][**ci**][**eager**][**documentation**][**interface**] Delete maxpool failed test [5409](https://github.com/Oneflow-Inc/oneflow/pull/5409)
- [**enhancement**][**eager**][**api**] Add tensor grad assginment [5379](https://github.com/Oneflow-Inc/oneflow/pull/5379)
- [**enhancement**][**bug**][**eager**] fix-abs [5398](https://github.com/Oneflow-Inc/oneflow/pull/5398)
- [**enhancement**][**bug**][**eager**][**interface**] Fix bn track running stats [5393](https://github.com/Oneflow-Inc/oneflow/pull/5393)
- [**enhancement**][**bug**][**eager**] Support uint dtype of constant op [5396](https://github.com/Oneflow-Inc/oneflow/pull/5396)
- [**enhancement**][**bug**][**eager**][**documentation**][**interface**] Delete useless code upsample [5392](https://github.com/Oneflow-Inc/oneflow/pull/5392)
- [**enhancement**][**ci**][**eager**][**interface**] add flow.view [5301](https://github.com/Oneflow-Inc/oneflow/pull/5301)
- [**enhancement**][**bug**][**ci**][**eager**][**api**][**interface**] Add masked select module [5356](https://github.com/Oneflow-Inc/oneflow/pull/5356)
- [**bug**][**eager**][**interface**] Fix batchnorm backward bug [5602](https://github.com/Oneflow-Inc/oneflow/pull/5602)
- [**enhancement**][**eager**] Support weight_dacay(l2 actually) [5587](https://github.com/Oneflow-Inc/oneflow/pull/5587)
- [**feature**][**eager**][**documentation**][**api**] Add new autotest [5588](https://github.com/Oneflow-Inc/oneflow/pull/5588)
- [**enhancement**][**eager**][**documentation**][**api**] Dev fmod [5404](https://github.com/Oneflow-Inc/oneflow/pull/5404)
- [**feature**][**eager**] Support inplace add [5432](https://github.com/Oneflow-Inc/oneflow/pull/5432)
- [**feature**][**eager**][**interface**] Feat tensor stride property [5543](https://github.com/Oneflow-Inc/oneflow/pull/5543)
- [**enhancement**][**feature**][**eager**][**documentation**][**api**] Add flip module [5541](https://github.com/Oneflow-Inc/oneflow/pull/5541)
- [**feature**][**eager**] Feat module repr [5486](https://github.com/Oneflow-Inc/oneflow/pull/5486)
- [**enhancement**][**bottleneck**][**bug**][**eager**][**interface**] Fix maxpool1d params [5493](https://github.com/Oneflow-Inc/oneflow/pull/5493)
- [**enhancement**][**feature**][**eager**][**interface**] Dev flow.utils.data part1 [5406](https://github.com/Oneflow-Inc/oneflow/pull/5406)
- [**bug**][**eager**][**api**] Fix tensor getitem bug [5474](https://github.com/Oneflow-Inc/oneflow/pull/5474)
- [**enhancement**][**eager**][**need-simple-ci**] export datasets interface [5691](https://github.com/Oneflow-Inc/oneflow/pull/5691)
- [**enhancement**][**eager**][**system**] rebase [5601](https://github.com/Oneflow-Inc/oneflow/pull/5601)
- [**enhancement**][**eager**][**test**] added nn.RecordBytesDecoder with its test [5475](https://github.com/Oneflow-Inc/oneflow/pull/5475)
- [**enhancement**][**feature**][**eager**][**need-simple-ci**] 0-dim tensor support [5552](https://github.com/Oneflow-Inc/oneflow/pull/5552)
- [**enhancement**][**bug**][**eager**] rewrite slice_update backward [5677](https://github.com/Oneflow-Inc/oneflow/pull/5677)
- [**enhancement**][**bug**][**eager**][**interface**] align view input style with torch [5676](https://github.com/Oneflow-Inc/oneflow/pull/5676)
- [**enhancement**][**eager**][**interface**][**need-simple-ci**] add autotests for modules [5666](https://github.com/Oneflow-Inc/oneflow/pull/5666)
- [**enhancement**][**bottleneck**][**eager**][**interface**] Dev constantpad1d op [5579](https://github.com/Oneflow-Inc/oneflow/pull/5579)
- [**enhancement**][**eager**][**api**][**interface**] Restruct MathOps AutoTest [5654](https://github.com/Oneflow-Inc/oneflow/pull/5654)
- [**enhancement**][**bug**][**ci**][**eager**] Fix flip bug [5657](https://github.com/Oneflow-Inc/oneflow/pull/5657)
- [**bug**][**eager**][**api**][**interface**] Fix expand module bug [5650](https://github.com/Oneflow-Inc/oneflow/pull/5650)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**] Fix repeat bug [5633](https://github.com/Oneflow-Inc/oneflow/pull/5633)
- [**enhancement**][**eager**][**test**][**api**][**interface**] Add new autotest [5617](https://github.com/Oneflow-Inc/oneflow/pull/5617)
- [**enhancement**][**eager**][**api**][**interface**] Dev flow.utils.data part2 [5500](https://github.com/Oneflow-Inc/oneflow/pull/5500)
- [**enhancement**][**bug**][**eager**] make setitem device match [5835](https://github.com/Oneflow-Inc/oneflow/pull/5835)
- [**bug**][**eager**][**api**][**interface**] align reshape input param with pytorch [5804](https://github.com/Oneflow-Inc/oneflow/pull/5804)
- [**feature**][**bug**][**eager**][**api**] Align where op with torch [5850](https://github.com/Oneflow-Inc/oneflow/pull/5850)
- [**enhancement**][**bug**][**eager**][**api**] Restruct prelu op [5829](https://github.com/Oneflow-Inc/oneflow/pull/5829)
- [**bug**][**eager**][**need-simple-ci**] fix pooling ceil_mode bug [5818](https://github.com/Oneflow-Inc/oneflow/pull/5818)
- [**enhancement**][**eager**] stateful local kernel supports consistent [5789](https://github.com/Oneflow-Inc/oneflow/pull/5789)
- [**bug**][**eager**][**api**][**interface**] Fix argwhere bug [5816](https://github.com/Oneflow-Inc/oneflow/pull/5816)
- [**enhancement**][**eager**][**documentation**][**api**] dev-nonzero [5809](https://github.com/Oneflow-Inc/oneflow/pull/5809)
- [**enhancement**][**feature**][**eager**][**api**] Add fake quantize op [5690](https://github.com/Oneflow-Inc/oneflow/pull/5690)
- [**enhancement**][**bug**][**eager**][**documentation**][**api**] Add api [5663](https://github.com/Oneflow-Inc/oneflow/pull/5663)
- [**enhancement**][**eager**] Refactor consistent infer result [5790](https://github.com/Oneflow-Inc/oneflow/pull/5790)
- [**bug**][**eager**][**need-simple-ci**] skip dataloader test [5780](https://github.com/Oneflow-Inc/oneflow/pull/5780)
- [**bug**][**eager**][**need-simple-ci**] fix 0-dim tensor.fill\_ [5771](https://github.com/Oneflow-Inc/oneflow/pull/5771)
- [**enhancement**][**eager**] Cpu mpi broadcast [5726](https://github.com/Oneflow-Inc/oneflow/pull/5726)
- [**feature**][**eager**] Feat grad mode classes [5956](https://github.com/Oneflow-Inc/oneflow/pull/5956)
- [**enhancement**][**bug**][**eager**] fix wrong names [5951](https://github.com/Oneflow-Inc/oneflow/pull/5951)
- [**enhancement**][**eager**][**system**] Local dep object pool [5953](https://github.com/Oneflow-Inc/oneflow/pull/5953)
- [**enhancement**][**eager**][**interface**] rename OpExprInterpState to AutoGradCaptureState [5918](https://github.com/Oneflow-Inc/oneflow/pull/5918)
- [**bug**][**eager**] Fix linear bug [5945](https://github.com/Oneflow-Inc/oneflow/pull/5945)
- [**bug**][**eager**] Fix tensor_meta update bug [5924](https://github.com/Oneflow-Inc/oneflow/pull/5924)
- [**enhancement**][**eager**] use flow.randperm [5928](https://github.com/Oneflow-Inc/oneflow/pull/5928)
- [**enhancement**][**eager**] consistent init/save/load [5896](https://github.com/Oneflow-Inc/oneflow/pull/5896)
- [**enhancement**][**bug**][**eager**][**documentation**][**interface**] Restruct sort and argsort op [5911](https://github.com/Oneflow-Inc/oneflow/pull/5911)
- [**enhancement**][**bug**][**eager**][**interface**] Try to fix the problem that the insightface cannot converge。 [5906](https://github.com/Oneflow-Inc/oneflow/pull/5906)
- [**enhancement**][**bug**][**eager**][**interface**] Add autotest [5899](https://github.com/Oneflow-Inc/oneflow/pull/5899)
- [**enhancement**][**eager**] The scheduler thread joins worker threads [5893](https://github.com/Oneflow-Inc/oneflow/pull/5893)
- [**enhancement**][**eager**] Bugfix async callback [5881](https://github.com/Oneflow-Inc/oneflow/pull/5881)
- [**feature**][**eager**] Feat tensor to bool [5836](https://github.com/Oneflow-Inc/oneflow/pull/5836)
- [**bug**][**eager**] Remove inplace broadcast_add [5551](https://github.com/Oneflow-Inc/oneflow/pull/5551)
- [**enhancement**][**eager**] Broadcast consistent shape and dtype [5784](https://github.com/Oneflow-Inc/oneflow/pull/5784)
- [**enhancement**][**eager**] Fix optimizer list parameters input bug [5848](https://github.com/Oneflow-Inc/oneflow/pull/5848)
- [**enhancement**][**eager**][**interface**] Dev flow.utils.data part3 [5644](https://github.com/Oneflow-Inc/oneflow/pull/5644)
- [**enhancement**][**eager**][**api**] Normalize naming of modules [6066](https://github.com/Oneflow-Inc/oneflow/pull/6066)
- [**enhancement**][**feature**][**eager**][**api**][**interface**] add trunc*normal* [6051](https://github.com/Oneflow-Inc/oneflow/pull/6051)
- [**enhancement**][**bug**][**eager**] AutoMatedTest support test module.parameter.grad [6043](https://github.com/Oneflow-Inc/oneflow/pull/6043)
- [**enhancement**][**feature**][**bug**][**eager**] add module **call** kwags [6069](https://github.com/Oneflow-Inc/oneflow/pull/6069)
- [**enhancement**][**eager**][**api**][**interface**] add tensor.item tensor.tolist [6021](https://github.com/Oneflow-Inc/oneflow/pull/6021)
- [**enhancement**][**eager**][**api**][**interface**] Export pool ops api [6047](https://github.com/Oneflow-Inc/oneflow/pull/6047)
- [**enhancement**][**bug**][**eager**][**test**][**documentation**][**interface**] Add more autotest sample [6039](https://github.com/Oneflow-Inc/oneflow/pull/6039)
- [**enhancement**][**bug**][**eager**][**system**] disable cuda_h2d stream [6020](https://github.com/Oneflow-Inc/oneflow/pull/6020)
- [**feature**][**eager**][**test**][**api**][**interface**] Add autotest codegen [6019](https://github.com/Oneflow-Inc/oneflow/pull/6019)
- [**feature**][**eager**][**documentation**] Refactor cosine lr scheduler [6000](https://github.com/Oneflow-Inc/oneflow/pull/6000)
- [**enhancement**][**eager**][**interface**] tensor.cpu/tensor.cuda [5894](https://github.com/Oneflow-Inc/oneflow/pull/5894)
- [**enhancement**][**eager**][**api**] Support consistent_tensor.to(dtype) [5991](https://github.com/Oneflow-Inc/oneflow/pull/5991)
- [**bug**][**eager**][**interface**] remove redundant codes in ModuleDict [5961](https://github.com/Oneflow-Inc/oneflow/pull/5961)
- [**bug**][**eager**] Fix LayerNorm check bug [6196](https://github.com/Oneflow-Inc/oneflow/pull/6196)
- [**enhancement**][**eager**][**api**] Change dropout api [6182](https://github.com/Oneflow-Inc/oneflow/pull/6182)
- [**enhancement**][**good for pr**][**eager**][**api**][**interface**] add: test convert dependency [6023](https://github.com/Oneflow-Inc/oneflow/pull/6023)
- [**enhancement**][**bug**][**eager**][**interface**] Fix autotest codegen bug [6171](https://github.com/Oneflow-Inc/oneflow/pull/6171)
- [**bug**][**eager**] restore instr_local_dep_object_pool_size for nccl [6160](https://github.com/Oneflow-Inc/oneflow/pull/6160)
- [**enhancement**][**eager**][**api**][**interface**] Aligin pooling op functional api names with torch [6163](https://github.com/Oneflow-Inc/oneflow/pull/6163)
- [**feature**][**bug**][**eager**][**api**][**interface**] delete file [6162](https://github.com/Oneflow-Inc/oneflow/pull/6162)
- [**bug**][**eager**] Fix optim load_state_dict bug [6152](https://github.com/Oneflow-Inc/oneflow/pull/6152)
- [**enhancement**][**eager**][**api**] add is_training to dropout functor [6148](https://github.com/Oneflow-Inc/oneflow/pull/6148)
- [**enhancement**][**eager**] Decompose nd sbp boxing [5800](https://github.com/Oneflow-Inc/oneflow/pull/5800)
- [**enhancement**][**eager**] support consistent_tensor.to(copy=True) [6122](https://github.com/Oneflow-Inc/oneflow/pull/6122)
- [**feature**][**eager**] Static grad scaler [6135](https://github.com/Oneflow-Inc/oneflow/pull/6135)
- [**bug**][**eager**] Fix LayerNorm expr bug [6121](https://github.com/Oneflow-Inc/oneflow/pull/6121)
- [**bug**][**eager**][**api**] move numpy c api init in numpy.cpp, make np array contiguous before copying [6117](https://github.com/Oneflow-Inc/oneflow/pull/6117)
- [**enhancement**][**eager**][**refactor**] Remove params from ParamGroup getitem [6096](https://github.com/Oneflow-Inc/oneflow/pull/6096)
- [**enhancement**][**feature**][**eager**] Support tensor and optimizer serialization [6087](https://github.com/Oneflow-Inc/oneflow/pull/6087)
- [**enhancement**][**bug**][**eager**] fix bug about tensor str in nonsymmetric cast and getitem in consist… [6239](https://github.com/Oneflow-Inc/oneflow/pull/6239)
- [**enhancement**][**eager**] Cpu all reduce [5849](https://github.com/Oneflow-Inc/oneflow/pull/5849)
- [**feature**][**eager**] Support assign copy interface [6228](https://github.com/Oneflow-Inc/oneflow/pull/6228)
- [**enhancement**][**eager**][**api**][**interface**] Dev reconstruct pad ops [6223](https://github.com/Oneflow-Inc/oneflow/pull/6223)
- [**enhancement**][**eager**][**api**][**interface**] support flow.cuda.is_available [6124](https://github.com/Oneflow-Inc/oneflow/pull/6124)
- [**bug**][**eager**] make flow.\_C.local_all_reduce sync lanuched [6175](https://github.com/Oneflow-Inc/oneflow/pull/6175)
- [**enhancement**][**eager**] Rename flow to oneflow in user hint [6190](https://github.com/Oneflow-Inc/oneflow/pull/6190)
- [**bug**][**eager**][**tooling**][**test**][**api**][**interface**] Autotest generate input tensor [6206](https://github.com/Oneflow-Inc/oneflow/pull/6206)
- [**enhancement**][**eager**] consistent tensor zeros\_() [6202](https://github.com/Oneflow-Inc/oneflow/pull/6202)
- [**enhancement**][**eager**] Cpu mpi [5865](https://github.com/Oneflow-Inc/oneflow/pull/5865)

Build enhancements:

- [**bug**][**build**] Fix GRPC compilation failure on CMake 3.20 [5255](https://github.com/Oneflow-Inc/oneflow/pull/5255)
- [**bug**][**build**] Refine header file copy [5254](https://github.com/Oneflow-Inc/oneflow/pull/5254)
- [**bug**][**build**] Fix older version CMake doesn't support multiple targets in CLI [5248](https://github.com/Oneflow-Inc/oneflow/pull/5248)
- [**bug**][**build**] Turn off NCCL_STATIC/CUDNN_STATIC when CUDA_STATIC is OFF [5243](https://github.com/Oneflow-Inc/oneflow/pull/5243)
- [**feature**][**build**] Fix support for Ninja and add Ninja build in Simple CI [5236](https://github.com/Oneflow-Inc/oneflow/pull/5236)
- [**enhancement**][**build**] Add cmake option CUDA_STATIC [5164](https://github.com/Oneflow-Inc/oneflow/pull/5164)
- [**bug**][**build**] Fix protobuf debug postfix [5233](https://github.com/Oneflow-Inc/oneflow/pull/5233)
- [**enhancement**][**ci**][**build**] Move default third party dir into build dir [5230](https://github.com/Oneflow-Inc/oneflow/pull/5230)
- [**enhancement**][**build**] Refine protobuf cmake [5216](https://github.com/Oneflow-Inc/oneflow/pull/5216)
- [**enhancement**][**ci**][**build**] Remove transport test main [5215](https://github.com/Oneflow-Inc/oneflow/pull/5215)
- [**enhancement**][**ci**][**build**] Speedup opencv build [5213](https://github.com/Oneflow-Inc/oneflow/pull/5213)
- [**enhancement**][**build**] Support clang [5015](https://github.com/Oneflow-Inc/oneflow/pull/5015)
- [**enhancement**][**documentation**][**build**] Add prefix when creating git archive [5201](https://github.com/Oneflow-Inc/oneflow/pull/5201)
- [**enhancement**][**build**] Add cmake option NCCL_STATIC [5160](https://github.com/Oneflow-Inc/oneflow/pull/5160)
- [**enhancement**][**build**] Refine CMake CUDA version handling [5192](https://github.com/Oneflow-Inc/oneflow/pull/5192)
- [**enhancement**][**build**] Use clang plugin to check Maybe variables are used [5358](https://github.com/Oneflow-Inc/oneflow/pull/5358)
- [**enhancement**][**build**] Add BUILD_BYPRODUCTS for ExternalProject_Add [5316](https://github.com/Oneflow-Inc/oneflow/pull/5316)
- [**enhancement**][**build**] Add cmake init cache to simplify user onboarding [5311](https://github.com/Oneflow-Inc/oneflow/pull/5311)
- [**feature**][**bug**][**build**] Fix macOS support and run macOS build in Simple CI [4947](https://github.com/Oneflow-Inc/oneflow/pull/4947)
- [**enhancement**][**build**] flatbuffers use mirror [5295](https://github.com/Oneflow-Inc/oneflow/pull/5295)
- [**enhancement**][**build**] Don't build test by default [5302](https://github.com/Oneflow-Inc/oneflow/pull/5302)
- [**enhancement**][**build**] Prevent building from scratch when toggle flag BUILD_GIT_VERSION [5259](https://github.com/Oneflow-Inc/oneflow/pull/5259)
- [**enhancement**][**build**] Refine gRPC, glog, gflags cmake for conda [5276](https://github.com/Oneflow-Inc/oneflow/pull/5276)
- [**feature**][**build**] Support XLA with CPU-only [5260](https://github.com/Oneflow-Inc/oneflow/pull/5260)
- [**enhancement**][**ci**][**onnx**][**build**] Remove ONNX from CI [5257](https://github.com/Oneflow-Inc/oneflow/pull/5257)
- [**enhancement**][**build**] Refactor build_wheel to support oneflowinc images [5427](https://github.com/Oneflow-Inc/oneflow/pull/5427)
- [**enhancement**][**build**] Add arg skip_audit in build wheel [5423](https://github.com/Oneflow-Inc/oneflow/pull/5423)
- [**bug**][**build**] hwloc disable shared [5388](https://github.com/Oneflow-Inc/oneflow/pull/5388)
- [**documentation**][**build**] Update readme for autoconf and libtool [5376](https://github.com/Oneflow-Inc/oneflow/pull/5376)
- [**enhancement**][**build**] remove dir python and compatible_single_client_python [5609](https://github.com/Oneflow-Inc/oneflow/pull/5609)
- [**bug**][**build**][**system**] Fix pyyaml version [5594](https://github.com/Oneflow-Inc/oneflow/pull/5594)
- [**enhancement**][**ci**][**build**] force release flags [5574](https://github.com/Oneflow-Inc/oneflow/pull/5574)
- [**bug**][**build**] prevent endless loop [5534](https://github.com/Oneflow-Inc/oneflow/pull/5534)
- [**enhancement**][**build**] Support sccache [5528](https://github.com/Oneflow-Inc/oneflow/pull/5528)
- [**enhancement**][**build**] Add definition for CMAKE_BUILD_TYPE and print cmake_build_type in oneflow doctor [5505](https://github.com/Oneflow-Inc/oneflow/pull/5505)
- [**enhancement**][**ci**][**build**][**need-simple-ci**] Fix macOS for recent changes [5705](https://github.com/Oneflow-Inc/oneflow/pull/5705)
- [**bug**][**build**] fix return type error on gcc 4.8.5 [5660](https://github.com/Oneflow-Inc/oneflow/pull/5660)
- [**enhancement**][**build**] Check CMAKE_BUILD_TYPE [5656](https://github.com/Oneflow-Inc/oneflow/pull/5656)
- [**enhancement**][**build**] add -Werror=return-type [5655](https://github.com/Oneflow-Inc/oneflow/pull/5655)
- [**enhancement**][**build**] Clean and fix for new py dir [5618](https://github.com/Oneflow-Inc/oneflow/pull/5618)
- [**enhancement**][**build**] cmake: disable array-bounds check & treat warnings as errors for pyextobj and oneflow_internal & fix warnings [5838](https://github.com/Oneflow-Inc/oneflow/pull/5838)
- [**bug**][**build**] set CMAKE_BUILD_TYPE to Release if undefined [5842](https://github.com/Oneflow-Inc/oneflow/pull/5842)
- [**enhancement**][**build**][**need-simple-ci**] Fix all warnings & Add option TREAT_WARING_AS_ERROR to cmake [5751](https://github.com/Oneflow-Inc/oneflow/pull/5751)
- [**enhancement**][**build**] add CMAKE_INTERPROCEDURAL_OPTIMIZATION in fast cmake cache [5970](https://github.com/Oneflow-Inc/oneflow/pull/5970)
- [**enhancement**][**build**] add clang tidy target [5957](https://github.com/Oneflow-Inc/oneflow/pull/5957)
- [**bug**][**build**] cmake: fix cmake cache args in opencv [5959](https://github.com/Oneflow-Inc/oneflow/pull/5959)
- [**enhancement**][**build**] Add cmake option USE_SYSTEM_NCCL [5897](https://github.com/Oneflow-Inc/oneflow/pull/5897)
- [**enhancement**][**build**] cmake: include third party headers as system headers to avoid warnings [5879](https://github.com/Oneflow-Inc/oneflow/pull/5879)
- [**enhancement**][**build**] Ignore opencv-python on machine aarch64 [5884](https://github.com/Oneflow-Inc/oneflow/pull/5884)
- [**enhancement**][**build**] enable CMake first class cuda support [5858](https://github.com/Oneflow-Inc/oneflow/pull/5858)
- [**bug**][**build**] Fix compile warning (strict-aliasing) [5872](https://github.com/Oneflow-Inc/oneflow/pull/5872)
- [**enhancement**][**bug**][**build**][**need-simple-ci**] Upgrade gtest and fix some errors raised by clang [6079](https://github.com/Oneflow-Inc/oneflow/pull/6079)
- [**bug**][**ci**][**build**] cmake: fix ninja build in CI [6072](https://github.com/Oneflow-Inc/oneflow/pull/6072)
- [**bug**][**build**] fix files not actually removed when building for multiple python versions [6060](https://github.com/Oneflow-Inc/oneflow/pull/6060)
- [**bug**][**build**][**api**] functional_api: fix build error in mac os [6010](https://github.com/Oneflow-Inc/oneflow/pull/6010)
- [**bug**][**build**][**need-simple-ci**][**need-single-client-tests**] Fix recompile from scratch [6036](https://github.com/Oneflow-Inc/oneflow/pull/6036)
- [**bug**][**build**] Turn on NVCC's warnings [6011](https://github.com/Oneflow-Inc/oneflow/pull/6011)
- [**bug**][**build**][**need-single-client-tests**] fix bundle .so of other python version [6034](https://github.com/Oneflow-Inc/oneflow/pull/6034)
- [**bug**][**ci**][**build**][**need-single-client-tests**] use copy_all_files_in_dir to replace copy_files [6033](https://github.com/Oneflow-Inc/oneflow/pull/6033)
- [**enhancement**][**build**] check compiler version in cmake [6026](https://github.com/Oneflow-Inc/oneflow/pull/6026)
- [**enhancement**][**build**] Add CUDA_NVCC_THREADS_NUMBER [6017](https://github.com/Oneflow-Inc/oneflow/pull/6017)
- [**enhancement**][**build**][**need-simple-ci**] optimize of_include_copy [5978](https://github.com/Oneflow-Inc/oneflow/pull/5978)
- [**enhancement**][**ci**][**build**][**need-single-client-tests**] CI: remove `-DTREAT_WARNINGS_AS_ERRORS=OFF` [6008](https://github.com/Oneflow-Inc/oneflow/pull/6008)
- [**enhancement**][**build**][**xla**] xrt: fix all warnings [5915](https://github.com/Oneflow-Inc/oneflow/pull/5915)
- [**enhancement**][**build**] Prevent opencv compile failure with std 17 [5997](https://github.com/Oneflow-Inc/oneflow/pull/5997)
- [**enhancement**][**build**] Use bundled cub [5998](https://github.com/Oneflow-Inc/oneflow/pull/5998)
- [**enhancement**][**ci**][**build**] update clang tidy diff warnings-as-errors option [5989](https://github.com/Oneflow-Inc/oneflow/pull/5989)
- [**enhancement**][**build**] Update run_clang_tidy.py to set return code and add warning-as-errors [5977](https://github.com/Oneflow-Inc/oneflow/pull/5977)
- [**enhancement**][**build**] check: fix clang-tidy-diff commands [5972](https://github.com/Oneflow-Inc/oneflow/pull/5972)
- [**bug**][**build**] Suppress NVCC warning 177-D [6094](https://github.com/Oneflow-Inc/oneflow/pull/6094)

XLA enhancements:

- [**bug**][**xla**] Make the blob header memory aligned. [5286](https://github.com/Oneflow-Inc/oneflow/pull/5286)

System:

- [**enhancement**][**system**] Refactor Memory Zone [5072](https://github.com/Oneflow-Inc/oneflow/pull/5072)
- [**enhancement**][**system**] Add interface InferContext::OutputTensorDesc [5219](https://github.com/Oneflow-Inc/oneflow/pull/5219)
- [**bug**][**system**] Lazy construct functor to make sure that the operators has already been registered. [5225](https://github.com/Oneflow-Inc/oneflow/pull/5225)
- [**enhancement**][**system**] Refactor infer ctx output isdynamic [5220](https://github.com/Oneflow-Inc/oneflow/pull/5220)
- [**enhancement**][**system**] Refactor infer ctx input isdynamic [5211](https://github.com/Oneflow-Inc/oneflow/pull/5211)
- [**enhancement**][**system**] Wake up the heartbeat thread immediately [5081](https://github.com/Oneflow-Inc/oneflow/pull/5081)
- [**enhancement**][**system**] Fix xla test case fail [5203](https://github.com/Oneflow-Inc/oneflow/pull/5203)
- [**enhancement**][**system**] Add interface InferContext::InputDType [5153](https://github.com/Oneflow-Inc/oneflow/pull/5153)
- [**purge**][**system**] delete const_cast in Output [5196](https://github.com/Oneflow-Inc/oneflow/pull/5196)
- [**feature**][**system**] Add hwloc for topology detection [5291](https://github.com/Oneflow-Inc/oneflow/pull/5291)
- [**enhancement**][**system**] fix registry may segment [5336](https://github.com/Oneflow-Inc/oneflow/pull/5336)
- [**enhancement**][**system**] Use functional api instead of op_expr_helper::XXXOp. [5364](https://github.com/Oneflow-Inc/oneflow/pull/5364)
- [**enhancement**][**system**] move btob to op [5274](https://github.com/Oneflow-Inc/oneflow/pull/5274)
- [**documentation**][**system**] Add Latest News section in README [5361](https://github.com/Oneflow-Inc/oneflow/pull/5361)
- [**enhancement**][**bug**][**system**] fix dropout module: return directly if not training [5346](https://github.com/Oneflow-Inc/oneflow/pull/5346)
- [**bug**][**system**] add missing JUST [5357](https://github.com/Oneflow-Inc/oneflow/pull/5357)
- [**documentation**][**system**] Add more communication outlets on README [5359](https://github.com/Oneflow-Inc/oneflow/pull/5359)
- [**enhancement**][**feature**][**system**] CommNet dynamic register memory [5281](https://github.com/Oneflow-Inc/oneflow/pull/5281)
- [**enhancement**][**system**] Use symbol device [5341](https://github.com/Oneflow-Inc/oneflow/pull/5341)
- [**enhancement**][**system**] fix multithread bug in env [5283](https://github.com/Oneflow-Inc/oneflow/pull/5283)
- [**bug**][**system**][**api**] fix bug in cfg_replacement [5335](https://github.com/Oneflow-Inc/oneflow/pull/5335)
- [**bug**][**system**] Fix create log directory thread-unsafe [5326](https://github.com/Oneflow-Inc/oneflow/pull/5326)
- [**bug**][**system**] fix_bug_in_make_parallel [5328](https://github.com/Oneflow-Inc/oneflow/pull/5328)
- [**enhancement**][**system**][**cfg**] replace train_conf, job_conf using cfg::xx [5263](https://github.com/Oneflow-Inc/oneflow/pull/5263)
- [**enhancement**][**system**][**quantization**] support tensorrt in qat [5287](https://github.com/Oneflow-Inc/oneflow/pull/5287)
- [**enhancement**][**system**][**api**] Export functional apis for oneflow.experimental. [5313](https://github.com/Oneflow-Inc/oneflow/pull/5313)
- [**enhancement**][**system**] fix bug check between cfg enum and proto enum [5285](https://github.com/Oneflow-Inc/oneflow/pull/5285)
- [**enhancement**][**system**] replace CHECK_EQ using CHECK_EQ_OR_RETURN [5279](https://github.com/Oneflow-Inc/oneflow/pull/5279)
- [**enhancement**][**system**] Refactor SbpXXX to cfg::SbpXXX [5120](https://github.com/Oneflow-Inc/oneflow/pull/5120)
- [**enhancement**][**system**][**api**] add detach for LazyMirroredtensorImpl [5270](https://github.com/Oneflow-Inc/oneflow/pull/5270)
- [**enhancement**][**system**] shorten XXIsDynamic4ArgNameAndIndex to be xxIsDynamic [5265](https://github.com/Oneflow-Inc/oneflow/pull/5265)
- [**enhancement**][**system**][**cfg**] job_config to cfg [5235](https://github.com/Oneflow-Inc/oneflow/pull/5235)
- [**feature**][**system**] Multi-Client LogicalRun degenerate to PhysicalRun [5479](https://github.com/Oneflow-Inc/oneflow/pull/5479)
- [**enhancement**][**system**] fix ConstructOp without JUST [5480](https://github.com/Oneflow-Inc/oneflow/pull/5480)
- [**enhancement**][**system**] Output arg modifier return maybe part 1 [5451](https://github.com/Oneflow-Inc/oneflow/pull/5451)
- [**feature**][**system**][**interface**] Fea/nn graph/graph build ctx [5420](https://github.com/Oneflow-Inc/oneflow/pull/5420)
- [**enhancement**][**system**] Throw exception if check failed [5457](https://github.com/Oneflow-Inc/oneflow/pull/5457)
- [**feature**][**system**] multi client launch [5372](https://github.com/Oneflow-Inc/oneflow/pull/5372)
- [**enhancement**][**system**][**api**] Optimize reduce mean [5452](https://github.com/Oneflow-Inc/oneflow/pull/5452)
- [**enhancement**][**system**] export Tensor only to python [5440](https://github.com/Oneflow-Inc/oneflow/pull/5440)
- [**enhancement**][**system**] Output arg modifier return maybe part_0 [5447](https://github.com/Oneflow-Inc/oneflow/pull/5447)
- [**enhancement**][**system**] ThreadMgr support AddPlan [5450](https://github.com/Oneflow-Inc/oneflow/pull/5450)
- [**enhancement**][**system**] Refactor infer ctx input tensordesc [5226](https://github.com/Oneflow-Inc/oneflow/pull/5226)
- [**enhancement**][**system**][**api**] instruction builder return maybe [5442](https://github.com/Oneflow-Inc/oneflow/pull/5442)
- [**feature**][**system**][**interface**] MultiClientSessionContext [5421](https://github.com/Oneflow-Inc/oneflow/pull/5421)
- [**enhancement**][**feature**][**system**] add launcher, update multi client launch and exit [5414](https://github.com/Oneflow-Inc/oneflow/pull/5414)
- [**purge**][**system**][**refactor**] Remove IOConf [5419](https://github.com/Oneflow-Inc/oneflow/pull/5419)
- [**enhancement**][**system**] Dev refine generator [5426](https://github.com/Oneflow-Inc/oneflow/pull/5426)
- [**enhancement**][**system**] Support inplace operations [5204](https://github.com/Oneflow-Inc/oneflow/pull/5204)
- [**enhancement**][**system**][**refactor**] Dev refactor generator [5397](https://github.com/Oneflow-Inc/oneflow/pull/5397)
- [**enhancement**][**system**] Add new placement init func [5408](https://github.com/Oneflow-Inc/oneflow/pull/5408)
- [**enhancement**][**system**] NNGraphIf [5387](https://github.com/Oneflow-Inc/oneflow/pull/5387)
- [**enhancement**][**system**][**refactor**] Cast explicitily in unpack call to avoid confilt with Optional. [5380](https://github.com/Oneflow-Inc/oneflow/pull/5380)
- [**enhancement**][**system**][**interface**] [Random Generator] Part2: Migrate functional dropout [5378](https://github.com/Oneflow-Inc/oneflow/pull/5378)
- [**enhancement**][**system**] replace ForeignJobInstance using JobInstance [5374](https://github.com/Oneflow-Inc/oneflow/pull/5374)
- [**enhancement**][**system**][**refactor**] Speedup reshape module by 5x. [5381](https://github.com/Oneflow-Inc/oneflow/pull/5381)
- [**feature**][**system**][**interface**] [Random Generator] Part1: Dev random generator [5360](https://github.com/Oneflow-Inc/oneflow/pull/5360)
- [**enhancement**][**system**] Add ONEFLOW_STREAM_CUDA_EVENT_FLAG_BLOCKING_SYNC [5612](https://github.com/Oneflow-Inc/oneflow/pull/5612)
- [**enhancement**][**system**] [part2]Remove singleclient outdated api [5568](https://github.com/Oneflow-Inc/oneflow/pull/5568)
- [**feature**][**system**][**interface**] nn.Graph call and launch impl [5580](https://github.com/Oneflow-Inc/oneflow/pull/5580)
- [**enhancement**][**system**] remove outdated doctest api and "experimental_api" [5564](https://github.com/Oneflow-Inc/oneflow/pull/5564)
- [**feature**][**system**][**interface**] Register ForeignCallback and Watcher in Multi-Client [5591](https://github.com/Oneflow-Inc/oneflow/pull/5591)
- [**enhancement**][**system**] [Part-1]remove outdated api and files of multi-client on master branch [5556](https://github.com/Oneflow-Inc/oneflow/pull/5556)
- [**feature**][**system**][**interface**] LazyInterpret build LocalTensor if input is local [5582](https://github.com/Oneflow-Inc/oneflow/pull/5582)
- [**enhancement**][**system**] add job_pass MultiClientAutoSourceAndSinkTick [5507](https://github.com/Oneflow-Inc/oneflow/pull/5507)
- [**feature**][**system**] Fea/nn graph/optimizer [5533](https://github.com/Oneflow-Inc/oneflow/pull/5533)
- [**feature**][**system**][**interface**] New/CloseRuntimeBuffers and RunLazyJob impl [5571](https://github.com/Oneflow-Inc/oneflow/pull/5571)
- [**feature**][**system**][**refactor**][**interface**] NNGraph interface and implement for CompileAndRuntime [5558](https://github.com/Oneflow-Inc/oneflow/pull/5558)
- [**feature**][**system**] Fea/nn graph/forward graph [5516](https://github.com/Oneflow-Inc/oneflow/pull/5516)
- [**enhancement**][**system**] Lazy job stream type [5389](https://github.com/Oneflow-Inc/oneflow/pull/5389)
- [**enhancement**][**system**] Refactor single client autotick [5506](https://github.com/Oneflow-Inc/oneflow/pull/5506)
- [**enhancement**][**system**] replace underline using dot in single client [5547](https://github.com/Oneflow-Inc/oneflow/pull/5547)
- [**bug**][**system**] fix return type [5548](https://github.com/Oneflow-Inc/oneflow/pull/5548)
- [**feature**][**system**][**interface**] LazyInterpret for UserOpExpr [5544](https://github.com/Oneflow-Inc/oneflow/pull/5544)
- [**enhancement**][**system**] Add ProfilerStart/ProfilerStop API [5542](https://github.com/Oneflow-Inc/oneflow/pull/5542)
- [**feature**][**system**][**interface**] LazyInterpreter for FetchOutputOpExpr and set op parallel_distribution [5527](https://github.com/Oneflow-Inc/oneflow/pull/5527)
- [**enhancement**][**system**] Multi client push pull [5492](https://github.com/Oneflow-Inc/oneflow/pull/5492)
- [**enhancement**][**system**] registry_callback_fn return maybe [5456](https://github.com/Oneflow-Inc/oneflow/pull/5456)
- [**enhancement**][**system**] bw_gen_fn return maybe [5455](https://github.com/Oneflow-Inc/oneflow/pull/5455)
- [**enhancement**][**system**] gen_bw_fn return maybe [5454](https://github.com/Oneflow-Inc/oneflow/pull/5454)
- [**enhancement**][**system**] Compatible single client [5417](https://github.com/Oneflow-Inc/oneflow/pull/5417)
- [**feature**][**system**][**interface**] GlobalMultiClientEnv and refine EagerExecution [5523](https://github.com/Oneflow-Inc/oneflow/pull/5523)
- [**enhancement**][**system**] Job pass maybe system [5503](https://github.com/Oneflow-Inc/oneflow/pull/5503)
- [**enhancement**][**system**] Remove Plan::net_topo [5502](https://github.com/Oneflow-Inc/oneflow/pull/5502)
- [**feature**][**system**][**interface**] LazyInterpret for FeedVariableOpExpr [5490](https://github.com/Oneflow-Inc/oneflow/pull/5490)
- [**enhancement**][**system**] Input arg modifier return maybe [5453](https://github.com/Oneflow-Inc/oneflow/pull/5453)
- [**feature**][**system**][**interface**] Fea/nn graph/block scope [5498](https://github.com/Oneflow-Inc/oneflow/pull/5498)
- [**feature**][**system**] jit_fuse_cast_scale [5332](https://github.com/Oneflow-Inc/oneflow/pull/5332)
- [**enhancement**][**system**] Remove obsolete Profiler [5747](https://github.com/Oneflow-Inc/oneflow/pull/5747)
- [**enhancement**][**system**][**api**] Dev fix batch norm not stats [5733](https://github.com/Oneflow-Inc/oneflow/pull/5733)
- [**enhancement**][**system**] rename rpc_token to TransportToken [5735](https://github.com/Oneflow-Inc/oneflow/pull/5735)
- [**enhancement**][**system**][**api**] Refacotr maximum minimum py2cpp [5724](https://github.com/Oneflow-Inc/oneflow/pull/5724)
- [**enhancement**][**system**] Replace piece_id with comm_net_sequence_number [5731](https://github.com/Oneflow-Inc/oneflow/pull/5731)
- [**enhancement**][**system**] beautify stack frame [5686](https://github.com/Oneflow-Inc/oneflow/pull/5686)
- [**enhancement**][**system**] Add env ONEFLOW_KERNEL_DISABLE_BLOB_ACCESS_CHECKER [5728](https://github.com/Oneflow-Inc/oneflow/pull/5728)
- [**enhancement**][**system**] Add env ONEFLOW_THREAD_ENABLE_LOCAL_MESSAGE_QUEUE [5720](https://github.com/Oneflow-Inc/oneflow/pull/5720)
- [**enhancement**][**system**][**api**][**refactor**] Refactor functional sub, mul and div apis [5713](https://github.com/Oneflow-Inc/oneflow/pull/5713)
- [**feature**][**system**] ddp [5008](https://github.com/Oneflow-Inc/oneflow/pull/5008)
- [**enhancement**][**system**][**api**][**refactor**] Refactor functional matmul and add apis. [5697](https://github.com/Oneflow-Inc/oneflow/pull/5697)
- [**bug**][**system**] Fix ClearKV("plan") [5710](https://github.com/Oneflow-Inc/oneflow/pull/5710)
- [**enhancement**][**system**] Rename cpu to async cpu [5712](https://github.com/Oneflow-Inc/oneflow/pull/5712)
- [**enhancement**][**system**] Support tensor.to()/to_local() [5271](https://github.com/Oneflow-Inc/oneflow/pull/5271)
- [**feature**][**system**][**refactor**][**interface**] Multi-Runtime for multi nn.Graph [5683](https://github.com/Oneflow-Inc/oneflow/pull/5683)
- [**bug**][**system**][**refactor**] Add tag for Optional inplace constructor [5619](https://github.com/Oneflow-Inc/oneflow/pull/5619)
- [**enhancement**][**system**] Move Global<CommNet> to env scope [5670](https://github.com/Oneflow-Inc/oneflow/pull/5670)
- [**enhancement**][**system**] add JUST wrapper [5681](https://github.com/Oneflow-Inc/oneflow/pull/5681)
- [**enhancement**][**system**] New sync consistent meta info [5634](https://github.com/Oneflow-Inc/oneflow/pull/5634)
- [**enhancement**][**system**][**refactor**][**interface**] Refactor RuntimeCtx for multi-runtime [5664](https://github.com/Oneflow-Inc/oneflow/pull/5664)
- [**feature**][**system**][**interface**] Feat: memory shared between EagerTensor with VariableRegst [5649](https://github.com/Oneflow-Inc/oneflow/pull/5649)
- [**enhancement**][**system**] Use functional call directly instead of construct a module and then call-Add [5613](https://github.com/Oneflow-Inc/oneflow/pull/5613)
- [**enhancement**][**system**] disable eager_op consistent mode [5647](https://github.com/Oneflow-Inc/oneflow/pull/5647)
- [**enhancement**][**system**] add msg_penddin_list in ibverbs_qp to optimize qp_init_attr.cap.max_send_wr [5485](https://github.com/Oneflow-Inc/oneflow/pull/5485)
- [**enhancement**][**system**] IBVerbsCommNet add knobs [5626](https://github.com/Oneflow-Inc/oneflow/pull/5626)
- [**enhancement**][**system**] Prune python tensor [5596](https://github.com/Oneflow-Inc/oneflow/pull/5596)
- [**feature**][**system**][**interface**] Feat: LazyInterpret infer op / tensor ParallelDescScope [5625](https://github.com/Oneflow-Inc/oneflow/pull/5625)
- [**enhancement**][**system**] Replace src tick with with wait and send ids [5603](https://github.com/Oneflow-Inc/oneflow/pull/5603)
- [**enhancement**][**system**] Support symbol placement type in functional. [5627](https://github.com/Oneflow-Inc/oneflow/pull/5627)
- [**enhancement**][**system**][**api**][**refactor**][**interface**] Dev advanced indexing [5559](https://github.com/Oneflow-Inc/oneflow/pull/5559)
- [**enhancement**][**system**] Optimize maybe. [5839](https://github.com/Oneflow-Inc/oneflow/pull/5839)
- [**enhancement**][**system**] Decorator 4 disable recursive boxing call [5796](https://github.com/Oneflow-Inc/oneflow/pull/5796)
- [**enhancement**][**system**] add_eager_boxing_and_op_interpreter_dispatch_error_info [5819](https://github.com/Oneflow-Inc/oneflow/pull/5819)
- [**enhancement**][**system**] Kernel CUDA Graphs Support [5725](https://github.com/Oneflow-Inc/oneflow/pull/5725)
- [**bug**][**system**] Fix placement print bug [5853](https://github.com/Oneflow-Inc/oneflow/pull/5853)
- [**bug**][**system**] when error msg formatting fails, return error->DebugString [5844](https://github.com/Oneflow-Inc/oneflow/pull/5844)
- [**enhancement**][**system**][**refactor**] Rename variables named `*parallel_distribution*` to `*nd_sbp*` (1) [5815](https://github.com/Oneflow-Inc/oneflow/pull/5815)
- [**feature**][**system**][**interface**] Support Free EagerTensor caught in nn.Graph build [5777](https://github.com/Oneflow-Inc/oneflow/pull/5777)
- [**enhancement**][**system**] Reuse CUDA event / Refine BnInOp2Blob / Refine channel [5837](https://github.com/Oneflow-Inc/oneflow/pull/5837)
- [**enhancement**][**system**][**serving**] fix bug in AddInputOutputOpsPass: check existence of key in HashMap(inferface_lbi2scope_sym_id) [5653](https://github.com/Oneflow-Inc/oneflow/pull/5653)
- [**enhancement**][**system**][**api**] unpack_call: impl new `unpack_call_dispatcher` for better performance [5820](https://github.com/Oneflow-Inc/oneflow/pull/5820)
- [**feature**][**system**] Feat consistent tensor python constructor [5812](https://github.com/Oneflow-Inc/oneflow/pull/5812)
- [**feature**][**system**] Support 0shape tensor [5620](https://github.com/Oneflow-Inc/oneflow/pull/5620)
- [**documentation**][**system**] fix launcher description [5770](https://github.com/Oneflow-Inc/oneflow/pull/5770)
- [**feature**][**system**][**interface**] Multi-nn.Graph memory reuse by Chunk manager [5658](https://github.com/Oneflow-Inc/oneflow/pull/5658)
- [**bug**][**system**] Fix naive b2p error [5806](https://github.com/Oneflow-Inc/oneflow/pull/5806)
- [**enhancement**][**system**] set created generator with default rng seed [5801](https://github.com/Oneflow-Inc/oneflow/pull/5801)
- [**enhancement**][**system**] enhance_local_to_consistent [5761](https://github.com/Oneflow-Inc/oneflow/pull/5761)
- [**feature**][**system**] add flow.randn [5736](https://github.com/Oneflow-Inc/oneflow/pull/5736)
- [**enhancement**][**system**] Refactor hierarchical parallel cast autograd [5764](https://github.com/Oneflow-Inc/oneflow/pull/5764)
- [**enhancement**][**system**] Collective boxing executor add_plan delete_plan [5495](https://github.com/Oneflow-Inc/oneflow/pull/5495)
- [**enhancement**][**system**] Fix throw abort [5795](https://github.com/Oneflow-Inc/oneflow/pull/5795)
- [**enhancement**][**system**] DECORATE [5794](https://github.com/Oneflow-Inc/oneflow/pull/5794)
- [**enhancement**][**system**] Inferface eager boxing [5682](https://github.com/Oneflow-Inc/oneflow/pull/5682)
- [**enhancement**][**system**] extract_consistent_to_consistent_op_expr [5870](https://github.com/Oneflow-Inc/oneflow/pull/5870)
- [**enhancement**][**system**] disable backward pass consistent tensor meta check. [5871](https://github.com/Oneflow-Inc/oneflow/pull/5871)
- [**enhancement**][**system**] Add CudaStreamIndexGenerator::GenerateNamedStreamIndex [5940](https://github.com/Oneflow-Inc/oneflow/pull/5940)
- [**bug**][**system**] Only query PCI bus id when CUDA version >= 11 [5937](https://github.com/Oneflow-Inc/oneflow/pull/5937)
- [**enhancement**][**system**] maybe: add `JUST_MSG` and `CHECK_JUST_MSG` [5904](https://github.com/Oneflow-Inc/oneflow/pull/5904)
- [**bug**][**system**] Fix bug scalar [5950](https://github.com/Oneflow-Inc/oneflow/pull/5950)
- [**enhancement**][**system**] framework: fix rvalue reference warnings [5948](https://github.com/Oneflow-Inc/oneflow/pull/5948)
- [**purge**][**system**] Remove CudaWorkType [5942](https://github.com/Oneflow-Inc/oneflow/pull/5942)
- [**enhancement**][**system**] refactor_symbol [5941](https://github.com/Oneflow-Inc/oneflow/pull/5941)
- [**bug**][**system**] consistent_tensor_infer_cache: fix memory leak [5938](https://github.com/Oneflow-Inc/oneflow/pull/5938)
- [**feature**][**system**] support to print gpu [5936](https://github.com/Oneflow-Inc/oneflow/pull/5936)
- [**enhancement**][**system**] Bugfix static check [5935](https://github.com/Oneflow-Inc/oneflow/pull/5935)
- [**bug**][**system**] fix nccl_version log [5934](https://github.com/Oneflow-Inc/oneflow/pull/5934)
- [**bug**][**system**] Fix bug of multi-GPU train nn.Graph extra mem cost in rank 0 [5930](https://github.com/Oneflow-Inc/oneflow/pull/5930)
- [**enhancement**][**system**] Only gradient acc be scheduled in parallel. [5926](https://github.com/Oneflow-Inc/oneflow/pull/5926)
- [**enhancement**][**bug**][**system**] fix_ddp_bug_on_8_process [5929](https://github.com/Oneflow-Inc/oneflow/pull/5929)
- [**enhancement**][**system**] Fix bug error msg format [5866](https://github.com/Oneflow-Inc/oneflow/pull/5866)
- [**feature**][**system**] print consistent tensor data [5902](https://github.com/Oneflow-Inc/oneflow/pull/5902)
- [**bug**][**system**] Move parse env to the constructor [5922](https://github.com/Oneflow-Inc/oneflow/pull/5922)
- [**enhancement**][**system**] Remove GlobalWorkStreamId/GlobalThrdId [5917](https://github.com/Oneflow-Inc/oneflow/pull/5917)
- [**bug**][**system**] shared_or_scalar: fix alias warnings [5916](https://github.com/Oneflow-Inc/oneflow/pull/5916)
- [**purge**][**system**] Remove CompActor [5919](https://github.com/Oneflow-Inc/oneflow/pull/5919)
- [**enhancement**][**system**] Use symbol dtype [5641](https://github.com/Oneflow-Inc/oneflow/pull/5641)
- [**enhancement**][**feature**][**system**] Control Graph / Session / Env's python c++ object destruction [5845](https://github.com/Oneflow-Inc/oneflow/pull/5845)
- [**enhancement**][**bug**][**system**] Sync access and assign indexing tensor. [5907](https://github.com/Oneflow-Inc/oneflow/pull/5907)
- [**enhancement**][**system**][**api**][**refactor**] Dev consistent arange [5883](https://github.com/Oneflow-Inc/oneflow/pull/5883)
- [**enhancement**][**system**] Lazy interpreter for new ConsistentToConsistentOpExpr [5903](https://github.com/Oneflow-Inc/oneflow/pull/5903)
- [**bug**][**system**] Fix BUG of LazyInterpret FreeEagerTensor memory shared with regst [5891](https://github.com/Oneflow-Inc/oneflow/pull/5891)
- [**bug**][**system**] fix typo in `raise RuntimeError` [5890](https://github.com/Oneflow-Inc/oneflow/pull/5890)
- [**enhancement**][**system**][**refactor**] Rename the `ParallelDistribution` class to `NdSbp` [5814](https://github.com/Oneflow-Inc/oneflow/pull/5814)
- [**feature**][**system**] add flow.rand [5722](https://github.com/Oneflow-Inc/oneflow/pull/5722)
- [**feature**][**system**] Lazy Interpret support infer default device cpu [5880](https://github.com/Oneflow-Inc/oneflow/pull/5880)
- [**enhancement**][**system**] Tensor str [5783](https://github.com/Oneflow-Inc/oneflow/pull/5783)
- [**feature**][**system**][**interface**] Lazy to_consistent [5774](https://github.com/Oneflow-Inc/oneflow/pull/5774)
- [**enhancement**][**system**] wait vm empty before exiting [5860](https://github.com/Oneflow-Inc/oneflow/pull/5860)
- [**enhancement**][**system**] Eager boxing n to 1 [5949](https://github.com/Oneflow-Inc/oneflow/pull/5949)
- [**enhancement**][**system**] add kernel observer [6052](https://github.com/Oneflow-Inc/oneflow/pull/6052)
- [**enhancement**][**ci**][**system**] Optimize ddp broadcast and add speed/memory test in ci [6044](https://github.com/Oneflow-Inc/oneflow/pull/6044)
- [**enhancement**][**system**] add var to control only print warning once when blocked [6045](https://github.com/Oneflow-Inc/oneflow/pull/6045)
- [**enhancement**][**system**][**refactor**] Rewrite pow and logical functional apis [6032](https://github.com/Oneflow-Inc/oneflow/pull/6032)
- [**enhancement**][**system**] Token seq id [5964](https://github.com/Oneflow-Inc/oneflow/pull/5964)
- [**enhancement**][**documentation**][**system**] Remove python function wrapper. [6012](https://github.com/Oneflow-Inc/oneflow/pull/6012)
- [**feature**][**system**] Add timeout and loc for blocking calls [6007](https://github.com/Oneflow-Inc/oneflow/pull/6007)
- [**enhancement**][**system**] Eager boxing 1 to n [5943](https://github.com/Oneflow-Inc/oneflow/pull/5943)
- [**enhancement**][**system**] Boxing expr [6015](https://github.com/Oneflow-Inc/oneflow/pull/6015)
- [**enhancement**][**system**] new_X_to_B [5987](https://github.com/Oneflow-Inc/oneflow/pull/5987)
- [**enhancement**][**system**] Add unimplemented return information [5952](https://github.com/Oneflow-Inc/oneflow/pull/5952)
- [**enhancement**][**system**] Revert "Faster decorator" [6006](https://github.com/Oneflow-Inc/oneflow/pull/6006)
- [**enhancement**][**system**] Throw exception if using advanced indexing for tensor setitem [6001](https://github.com/Oneflow-Inc/oneflow/pull/6001)
- [**enhancement**][**system**] Support eager boxing sm 2 sn [5869](https://github.com/Oneflow-Inc/oneflow/pull/5869)
- [**enhancement**][**system**] Move framework/local_dep_object.\* to the eager directory [5988](https://github.com/Oneflow-Inc/oneflow/pull/5988)
- [**enhancement**][**system**] Fix builtin op arg tuple. [5464](https://github.com/Oneflow-Inc/oneflow/pull/5464)
- [**feature**][**system**][**refactor**] Dev functional multiple signatures [5982](https://github.com/Oneflow-Inc/oneflow/pull/5982)
- [**enhancement**][**system**] Faster decorator [5996](https://github.com/Oneflow-Inc/oneflow/pull/5996)
- [**enhancement**][**system**] Placed nd sbp [5995](https://github.com/Oneflow-Inc/oneflow/pull/5995)
- [**feature**][**system**] Support asymmetric input/output/variable tensors in nn.Graph [5983](https://github.com/Oneflow-Inc/oneflow/pull/5983)
- [**enhancement**][**system**] LightActor [5868](https://github.com/Oneflow-Inc/oneflow/pull/5868)
- [**bug**][**system**] Prevent running oneflow in forked subprocess [5976](https://github.com/Oneflow-Inc/oneflow/pull/5976)
- [**bug**][**system**] common/error: fix build error in mac os [5971](https://github.com/Oneflow-Inc/oneflow/pull/5971)
- [**bug**][**system**] fix_bug_test_tensor_str [5958](https://github.com/Oneflow-Inc/oneflow/pull/5958)
- [**enhancement**][**system**] Refine StreamContext [6191](https://github.com/Oneflow-Inc/oneflow/pull/6191)
- [**enhancement**][**system**] container_util: fix VectorAt, remove useless MutMapAt [6172](https://github.com/Oneflow-Inc/oneflow/pull/6172)
- [**enhancement**][**system**] Typesafe KernelState [6198](https://github.com/Oneflow-Inc/oneflow/pull/6198)
- [**enhancement**][**system**] Primitive based copy task node [6195](https://github.com/Oneflow-Inc/oneflow/pull/6195)
- [**feature**][**system**][**interface**] Lazy support Scalar [6181](https://github.com/Oneflow-Inc/oneflow/pull/6181)
- [**enhancement**][**system**] Disable implicit boxing when parallel num eq one [6188](https://github.com/Oneflow-Inc/oneflow/pull/6188)
- [**enhancement**][**system**] Primitive [6183](https://github.com/Oneflow-Inc/oneflow/pull/6183)
- [**enhancement**][**system**] Remove IDMgr::GetGpuPhyIdFromThrdId/IDMgr::GetDeviceTypeFromThrdId [6169](https://github.com/Oneflow-Inc/oneflow/pull/6169)
- [**enhancement**][**system**] remove op_expr_helper inside gradient_funcs [6057](https://github.com/Oneflow-Inc/oneflow/pull/6057)
- [**feature**][**system**][**api**] Add tensor yaml, support export tensor functional api. [6099](https://github.com/Oneflow-Inc/oneflow/pull/6099)
- [**feature**][**system**] Plan memory log [6151](https://github.com/Oneflow-Inc/oneflow/pull/6151)
- [**feature**][**system**] Add dtype bfloat16 [5304](https://github.com/Oneflow-Inc/oneflow/pull/5304)
- [**enhancement**][**system**] StreamContext [6129](https://github.com/Oneflow-Inc/oneflow/pull/6129)
- [**bug**][**system**] Fix wrong inplace acc grad [6146](https://github.com/Oneflow-Inc/oneflow/pull/6146)
- [**enhancement**][**system**] UserKernel remove job_desc [6144](https://github.com/Oneflow-Inc/oneflow/pull/6144)
- [**enhancement**][**system**][**api**] Fea/graph/add outputs buffer to enable pipeline [6126](https://github.com/Oneflow-Inc/oneflow/pull/6126)
- [**enhancement**][**system**] not fuse request for nccl 2.10.3 [6136](https://github.com/Oneflow-Inc/oneflow/pull/6136)
- [**bug**][**system**] NewUniqueId thread safe [6141](https://github.com/Oneflow-Inc/oneflow/pull/6141)
- [**enhancement**][**system**] XRT remove job_desc [6139](https://github.com/Oneflow-Inc/oneflow/pull/6139)
- [**enhancement**][**system**] SystemOpFillJobNamePass [6138](https://github.com/Oneflow-Inc/oneflow/pull/6138)
- [**enhancement**][**system**] mv_boxing_folder_to_core [6140](https://github.com/Oneflow-Inc/oneflow/pull/6140)
- [**enhancement**][**system**] Refactor boxing interpreter to boxing expr [6134](https://github.com/Oneflow-Inc/oneflow/pull/6134)
- [**enhancement**][**system**] Eager boxing one to one [6048](https://github.com/Oneflow-Inc/oneflow/pull/6048)
- [**enhancement**][**system**] Vm cpu efficiency [6110](https://github.com/Oneflow-Inc/oneflow/pull/6110)
- [**enhancement**][**system**] Naive generic boxing [6116](https://github.com/Oneflow-Inc/oneflow/pull/6116)
- [**feature**][**system**] send/recv [5992](https://github.com/Oneflow-Inc/oneflow/pull/5992)
- [**enhancement**][**system**] disable_print_stack_in_tensor_numpy [6123](https://github.com/Oneflow-Inc/oneflow/pull/6123)
- [**feature**][**system**] add all_reduce by to_consistent [5963](https://github.com/Oneflow-Inc/oneflow/pull/5963)
- [**enhancement**][**system**] KernelContext [6084](https://github.com/Oneflow-Inc/oneflow/pull/6084)
- [**enhancement**][**bug**][**system**] Fix sync nccl and async nccl deadlock [6071](https://github.com/Oneflow-Inc/oneflow/pull/6071)
- [**bug**][**system**][**refactor**] Refactor to local [6098](https://github.com/Oneflow-Inc/oneflow/pull/6098)
- [**enhancement**][**system**] Replace xor with hash combine (part 1) [6078](https://github.com/Oneflow-Inc/oneflow/pull/6078)
- [**enhancement**][**system**] Optimize error message [6073](https://github.com/Oneflow-Inc/oneflow/pull/6073)
- [**enhancement**][**system**] Rename Error::xx to Error::xxError [6049](https://github.com/Oneflow-Inc/oneflow/pull/6049)
- [**enhancement**][**system**] send formatted msg to glog [5999](https://github.com/Oneflow-Inc/oneflow/pull/5999)
- [**feature**][**bottleneck**][**bug**][**system**][**interface**] [Feat.] NNGraph new eager tensor for new variable created in JobPass [6091](https://github.com/Oneflow-Inc/oneflow/pull/6091)
- [**bug**][**system**] Fix bug of multi-GPU eager copy D2H extra mem cost in rank 0 [6092](https://github.com/Oneflow-Inc/oneflow/pull/6092)
- [**enhancement**][**system**][**api**] Rename module flow.F to flow.\_C [6053](https://github.com/Oneflow-Inc/oneflow/pull/6053)
- [**feature**][**system**][**interface**] [Feat.] Eager consistent OFRecordReader [6089](https://github.com/Oneflow-Inc/oneflow/pull/6089)
- [**enhancement**][**system**][**api**] Dev fix and align interface [6075](https://github.com/Oneflow-Inc/oneflow/pull/6075)
- [**feature**][**bottleneck**][**bug**][**system**][**interface**] NNGraph input/output valid by register tensors [6240](https://github.com/Oneflow-Inc/oneflow/pull/6240)
- [**bug**][**system**][**interface**] Fix bug of Multi-Client src tick output order [6221](https://github.com/Oneflow-Inc/oneflow/pull/6221)
- [**enhancement**][**bug**][**system**] Add cast primitive [6234](https://github.com/Oneflow-Inc/oneflow/pull/6234)
- [**feature**][**bottleneck**][**system**][**interface**] Auto FixPipelineStageIdPass [6204](https://github.com/Oneflow-Inc/oneflow/pull/6204)
- [**enhancement**][**system**] move scalar to oneflow namespace. [6235](https://github.com/Oneflow-Inc/oneflow/pull/6235)
- [**enhancement**][**system**] UserKernel init CUDA Graphs with state [6230](https://github.com/Oneflow-Inc/oneflow/pull/6230)
- [**feature**][**system**] Comm broadcast [6213](https://github.com/Oneflow-Inc/oneflow/pull/6213)
- [**enhancement**][**system**][**refactor**] Rename op*name* to op*type_name* in AutogradEngine [6154](https://github.com/Oneflow-Inc/oneflow/pull/6154)
- [**enhancement**][**system**] Add memset primitive [6218](https://github.com/Oneflow-Inc/oneflow/pull/6218)
- [**enhancement**][**system**] Add StreamContext::device_type()/DeviceCtx::device_type() [6217](https://github.com/Oneflow-Inc/oneflow/pull/6217)
- [**feature**][**system**] add all_gather and fix bug of multi rank doctest [6189](https://github.com/Oneflow-Inc/oneflow/pull/6189)
- [**feature**][**system**][**interface**] [Feat.] Lazy interpreter skip hierarchical_parallel_cast [6208](https://github.com/Oneflow-Inc/oneflow/pull/6208)
- [**purge**][**system**] Cleanup KernelUtil [6212](https://github.com/Oneflow-Inc/oneflow/pull/6212)
- [**enhancement**][**system**] StreamContextAdapter [6205](https://github.com/Oneflow-Inc/oneflow/pull/6205)
- [**enhancement**][**system**] Dev eliminate gcc warnings [6199](https://github.com/Oneflow-Inc/oneflow/pull/6199)
- [**feature**][**bottleneck**][**system**][**interface**] [Feat.] nn.Graph support grad acc with input/output tensor [6155](https://github.com/Oneflow-Inc/oneflow/pull/6155)
- [**enhancement**][**system**] Cpu symetric s to s [6153](https://github.com/Oneflow-Inc/oneflow/pull/6153)
- [**enhancement**][**system**][**upload-core**] Op expr infer tensor meta [5064](https://github.com/Oneflow-Inc/oneflow/pull/5064)
- [**enhancement**][**system**] Infer consistent tensor meta [5362](https://github.com/Oneflow-Inc/oneflow/pull/5362)

CI enhancements:

- [**bug**][**ci**][**api**][**interface**] Refine module test [5232](https://github.com/Oneflow-Inc/oneflow/pull/5232)
- [**enhancement**][**ci**] Add Simple CI, runs CPU-only on GitHub hosted servers [5207](https://github.com/Oneflow-Inc/oneflow/pull/5207)
- [**enhancement**][**ci**] Run exe test in CPU-only [5202](https://github.com/Oneflow-Inc/oneflow/pull/5202)
- [**enhancement**][**ci**] Cancel all workflow runs but the latest [5206](https://github.com/Oneflow-Inc/oneflow/pull/5206)
- [**enhancement**][**ci**] Fix master not running Simple CI [5368](https://github.com/Oneflow-Inc/oneflow/pull/5368)
- [**enhancement**][**ci**] Refine Simple CI and Clang analysis [5367](https://github.com/Oneflow-Inc/oneflow/pull/5367)
- [**enhancement**][**feature**][**bug**][**ci**][**documentation**][**interface**] Fix upsample bilinear bug [5363](https://github.com/Oneflow-Inc/oneflow/pull/5363)
- [**enhancement**][**ci**] Build nightly for py39 [5318](https://github.com/Oneflow-Inc/oneflow/pull/5318)
- [**enhancement**][**ci**] Try distributed run for 3 times to prevent failure [5305](https://github.com/Oneflow-Inc/oneflow/pull/5305)
- [**enhancement**][**ci**] Upload Simple CI logs to cloud [5268](https://github.com/Oneflow-Inc/oneflow/pull/5268)
- [**enhancement**][**ci**] Remove cpu_op_eager and cuda_op_eager [5470](https://github.com/Oneflow-Inc/oneflow/pull/5470)
- [**bug**][**ci**] fix segfault in clang plugin [5437](https://github.com/Oneflow-Inc/oneflow/pull/5437)
- [**enhancement**][**ci**] Refine Simple CI error output [5435](https://github.com/Oneflow-Inc/oneflow/pull/5435)
- [**enhancement**][**ci**] Add conda env to Simple CI [5385](https://github.com/Oneflow-Inc/oneflow/pull/5385)
- [**enhancement**][**ci**] Fix clang plugin core file not found [5390](https://github.com/Oneflow-Inc/oneflow/pull/5390)
- [**bug**][**ci**] upload core when build with clang plugin [5384](https://github.com/Oneflow-Inc/oneflow/pull/5384)
- [**bug**][**ci**] clang plugin skip more files [5373](https://github.com/Oneflow-Inc/oneflow/pull/5373)
- [**enhancement**][**ci**] Use gh-action-scheduler-v2 [5370](https://github.com/Oneflow-Inc/oneflow/pull/5370)
- [**enhancement**][**ci**] relax speed threshold [5569](https://github.com/Oneflow-Inc/oneflow/pull/5569)
- [**bug**][**ci**] Fix wrong test path under compatible [5567](https://github.com/Oneflow-Inc/oneflow/pull/5567)
- [**enhancement**][**ci**][**need-simple-ci**] Prevent upload logs automatically [5560](https://github.com/Oneflow-Inc/oneflow/pull/5560)
- [**enhancement**][**ci**][**interface**] Add `nn.AdaptiveAvgPool1d` and `nn.AdaptiveAvgPool3d` [5445](https://github.com/Oneflow-Inc/oneflow/pull/5445)
- [**feature**][**ci**] add speed test in ci [5496](https://github.com/Oneflow-Inc/oneflow/pull/5496)
- [**enhancement**][**ci**] Reduce usage of Simple CI [5546](https://github.com/Oneflow-Inc/oneflow/pull/5546)
- [**feature**][**bug**][**ci**][**api**] Restruct upsample module [5524](https://github.com/Oneflow-Inc/oneflow/pull/5524)
- [**feature**][**ci**] multi client launcher test [5488](https://github.com/Oneflow-Inc/oneflow/pull/5488)
- [**enhancement**][**ci**] Remove automerge if cuda_new_interface failed [5519](https://github.com/Oneflow-Inc/oneflow/pull/5519)
- [**enhancement**][**ci**] Prevent adding subdir in python/test [5514](https://github.com/Oneflow-Inc/oneflow/pull/5514)
- [**enhancement**][**ci**] piprepo->pipindex [5517](https://github.com/Oneflow-Inc/oneflow/pull/5517)
- [**enhancement**][**ci**] add dynamic_loss_scale in ci tests [5337](https://github.com/Oneflow-Inc/oneflow/pull/5337)
- [**enhancement**][**ci**] Add timeout for wait_gpu_slot [5497](https://github.com/Oneflow-Inc/oneflow/pull/5497)
- [**enhancement**][**feature**][**ci**] new static check based on clang-tidy [5476](https://github.com/Oneflow-Inc/oneflow/pull/5476)
- [**enhancement**][**ci**] Fix url not downloadable in some browers [5701](https://github.com/Oneflow-Inc/oneflow/pull/5701)
- [**feature**][**ci**] multi client multi machine test [5685](https://github.com/Oneflow-Inc/oneflow/pull/5685)
- [**enhancement**][**ci**] Add cpu new interface CI [5639](https://github.com/Oneflow-Inc/oneflow/pull/5639)
- [**enhancement**][**ci**][**need-simple-ci**] Mv clangtidy to simple ci [5667](https://github.com/Oneflow-Inc/oneflow/pull/5667)
- [**enhancement**][**ci**][**need-simple-ci**] use clang tidy appimage in ci [5841](https://github.com/Oneflow-Inc/oneflow/pull/5841)
- [**enhancement**][**ci**] Use gcc 7 in release to prevent error [5840](https://github.com/Oneflow-Inc/oneflow/pull/5840)
- [**enhancement**][**ci**] bn tol 1e-4 => 1e-3 [5811](https://github.com/Oneflow-Inc/oneflow/pull/5811)
- [**enhancement**][**ci**] fix distributed run on built dir [5810](https://github.com/Oneflow-Inc/oneflow/pull/5810)
- [**enhancement**][**ci**] fix third party mirror check_sum [5802](https://github.com/Oneflow-Inc/oneflow/pull/5802)
- [**ci**][**documentation**] find more accurately which files need to be doctested [5782](https://github.com/Oneflow-Inc/oneflow/pull/5782)
- [**enhancement**][**ci**] Print stack unconditionally [5779](https://github.com/Oneflow-Inc/oneflow/pull/5779)
- [**enhancement**][**ci**][**need-simple-ci**] Enable more checkers for clang-tidy in CI [5738](https://github.com/Oneflow-Inc/oneflow/pull/5738)
- [**enhancement**][**ci**] CI: add clang-tidy check to test.yaml [5920](https://github.com/Oneflow-Inc/oneflow/pull/5920)
- [**ci**][**documentation**] fix docstring in oneflow.nn.functional namespace [5807](https://github.com/Oneflow-Inc/oneflow/pull/5807)
- [**enhancement**][**ci**] disable TREAT_WARNINGS_AS_ERRORS in Release CI [5886](https://github.com/Oneflow-Inc/oneflow/pull/5886)
- [**enhancement**][**ci**] Skip ci jobs by git diff [5863](https://github.com/Oneflow-Inc/oneflow/pull/5863)
- [**bug**][**ci**] quick fix 5978 [6030](https://github.com/Oneflow-Inc/oneflow/pull/6030)
- [**enhancement**][**bug**][**ci**] fix clang tidy diff options and file format [5990](https://github.com/Oneflow-Inc/oneflow/pull/5990)
- [**enhancement**][**ci**] add flow.relu [5847](https://github.com/Oneflow-Inc/oneflow/pull/5847)
- [**enhancement**][**ci**] equal => allclose [6164](https://github.com/Oneflow-Inc/oneflow/pull/6164)
- [**bug**][**ci**][**need-simple-ci**] CI: fix clang tidy checks in simple ci [6161](https://github.com/Oneflow-Inc/oneflow/pull/6161)
- [**enhancement**][**bug**][**ci**][**documentation**][**api**] add interpolate and layer_norm docs [6157](https://github.com/Oneflow-Inc/oneflow/pull/6157)
- [**bug**][**ci**] update speed test [6113](https://github.com/Oneflow-Inc/oneflow/pull/6113)
- [**enhancement**][**bug**][**ci**][**documentation**][**api**] speed import oneflow [6107](https://github.com/Oneflow-Inc/oneflow/pull/6107)
- [**bug**][**ci**] Also try install dev deps for CODEGEN_PYTHON_EXECUTABLE [6115](https://github.com/Oneflow-Inc/oneflow/pull/6115)
- [**bug**][**ci**][**need-simple-ci**] set gtest_CMAKE_DEBUG_POSTFIX "d" [6085](https://github.com/Oneflow-Inc/oneflow/pull/6085)
- [**enhancement**][**ci**] add cache init file for clang and CI build with clang [6062](https://github.com/Oneflow-Inc/oneflow/pull/6062)
- [**enhancement**][**ci**] add emoji in speed test output, make it continue-on-error [6214](https://github.com/Oneflow-Inc/oneflow/pull/6214)

Test enhancements:

- [**bug**][**test**][**interface**] Fix acos ci bug [5217](https://github.com/Oneflow-Inc/oneflow/pull/5217)
- [**feature**][**test**] implement automated test [5321](https://github.com/Oneflow-Inc/oneflow/pull/5321)
- [**enhancement**][**test**] move generator test into ops folder to accelerate tests [5472](https://github.com/Oneflow-Inc/oneflow/pull/5472)
- [**feature**][**test**][**api**] Add autotest part2 [5467](https://github.com/Oneflow-Inc/oneflow/pull/5467)
- [**enhancement**][**test**][**api**][**interface**] Add some tests with the new framework for auto testing [5561](https://github.com/Oneflow-Inc/oneflow/pull/5561)
- [**bug**][**test**] fix test error when do multi case test on graph [5590](https://github.com/Oneflow-Inc/oneflow/pull/5590)
- [**enhancement**][**test**] Refine module test using auto test by yaochi [5484](https://github.com/Oneflow-Inc/oneflow/pull/5484)
- [**enhancement**][**test**] Add autotest for BatchNorm2d [5734](https://github.com/Oneflow-Inc/oneflow/pull/5734)
- [**enhancement**][**test**] RTH_update_op_test [5823](https://github.com/Oneflow-Inc/oneflow/pull/5823)
- [**enhancement**][**test**] dev adamw graph config [5745](https://github.com/Oneflow-Inc/oneflow/pull/5745)
- [**feature**][**test**][**api**][**interface**] Add new autotest [5562](https://github.com/Oneflow-Inc/oneflow/pull/5562)
- [**bug**][**test**] restore test of alexnet graph [5798](https://github.com/Oneflow-Inc/oneflow/pull/5798)
- [**enhancement**][**test**][**interface**] add zhangshen op-test [5600](https://github.com/Oneflow-Inc/oneflow/pull/5600)
- [**feature**][**bug**][**tooling**][**test**][**interface**] Record autotest wrong code [5923](https://github.com/Oneflow-Inc/oneflow/pull/5923)
- [**enhancement**][**feature**][**test**][**api**] add randint [5718](https://github.com/Oneflow-Inc/oneflow/pull/5718)
- [**bug**][**test**] fix multi machine test [5984](https://github.com/Oneflow-Inc/oneflow/pull/5984)
- [**enhancement**][**test**][**interface**] some op test [6095](https://github.com/Oneflow-Inc/oneflow/pull/6095)

Tooling enhancements:

- [**bug**][**tooling**] user/summary: fix memory leak in `FillImageInSummary` [5742](https://github.com/Oneflow-Inc/oneflow/pull/5742)
- [**enhancement**][**tooling**][**cfg**] cfg: add move assignment operator for performance [5962](https://github.com/Oneflow-Inc/oneflow/pull/5962)
- [**enhancement**][**tooling**][**api**][**refactor**] refactor_all_device_placement_api [6080](https://github.com/Oneflow-Inc/oneflow/pull/6080)

Lazy 模式的功能更新

支持 2-D SBP
- 转为2维
python
with flow.scope.placement("gpu", "0:0-3", (2, 2)):
x = flow.hierarchical_parallel_cast(
x, parallel_distribution=["B", "S(1)"]
)

- 转为1维
python
with flow.scope.placement("gpu", "0:0-3", (4,)):
x = flow.hierarchical_parallel_cast(
x, parallel_distribution=["S(0)"]
)


支持流水并行的新接口
- 创建 `pipeline_stage` 的 scope
python
with flow.experimental.scope.config(
pipeline_stage_id_hint=dist_util.get_layer_stage(layer_idx)
):
...

- 为了是流水并行能更好的工作，必须使用梯度累加，可以使用有限内存跑更大 batch。通过 config 设置梯度累加的步数：

func_cfg = flow.FunctionConfig()
...
func_cfg.train.num_gradient_accumulation_steps(args.num_accumulation_steps)
flow.global_function(..., function_config=func_cfg)


支持 ZeRO 优化
- 开启方式：
python
func_cfg = flow.FunctionConfig()
...
func_cfg.optimizer_placement_optimization_mode(mode) mode = "non_distributed" or "distributed_split"
flow.global_function(..., function_config=func_cfg)

- 示例代码请参考这个[测试用例](https://github.com/Oneflow-Inc/oneflow/blob/master/oneflow/python/test/ops/test_optimizer_placement_optimization.py#L27)
- `mode = "distributed_split"` 对应 DeepSpeed ZeRO 优化的 stage 2

支持 Checkpointing 的新接口
python
with flow.experimental.scope.config(
checkpointing=True
):

欢迎阅读相关文章：[亚线性内存优化—activation checkpointing在oneflow中的实现](https://zhuanlan.zhihu.com/p/373662730)

Eager 模式的功能更新

提供`oneflow.experimental` 命名空间，部分对齐 `torch.xxx` 接口

- 新接口的使用方法
python
import oneflow.experimental as flow
flow.enable_eager_execution() 启用 eager


- 目前部分对齐的功能

python
flow.nn.Conv2d <-> torch.nn.Conv2d
flow.nn.BatchNorm2d <-> torch.nn.BatchNorm2d
flow.nn.ReLU <-> torch.nn.ReLU
flow.nn.MaxPool2d <-> torch.nn.MaxPool2d
flow.nn.AvgPool2d <-> torch.nn.AvgPool2d
flow.nn.Linear <-> torch.nn.Linear
flow.nn.CrossEntropyLoss <-> torch.nn.CrossEntropyLoss
flow.nn.Sequential <-> torch.nn.Sequential

flow.nn.Module.to <-> torch.nn.Module.to
flow.nn.Module.state_dict <-> torch.nn.Module.state_dict
flow.nn.Module.load_state_dict <-> torch.nn.Module.load_state_dict

flow.save <-> torch.save
flow.load <-> torch.load

flow.Tensor <-> torch.Tensor
flow.tensor <-> torch.tensor
flow.tensor.to <-> torch.tensor.to
flow.tensor.numpy <-> torch.tensor.numpy
flow.tensor 加减乘除 <-> torch.tensor 加减乘除
flow.tensor.flatten <-> torch.tensor.flatten
flow.tensor.softmax <-> torch.tensor.softmax

flow.optim.SGD <-> torch.optim.SGD

基于上述模块已经可以轻松搭建常用网络，如：ResNet、BERT、MobileNetV3 等。后续版本将对齐/支持更多接口，届时可将大多数基于 Pytorch 搭建的网络，轻松切换到 OneFlow。

- 快速上手例子 lenet: https://github.com/Oneflow-Inc/models/blob/main/quick_start_demo_lenet/lenet.py

- 新接口文档链接：https://oneflow.readthedocs.io/en/master/experimental.html

- 对齐 torch vision 的 ResNet50 示例代码：https://github.com/Oneflow-Inc/models/tree/main/resnet50

- 接下里的几个版本会增加更多 对齐 PyTorch 的接口

- `experimental` 下对齐的接口在 0.6.0 版本更新时会被移动到 oneflow 的命名空间下，届时会完全对齐 PyTorch，OneFlow 0.6.0 会将 eager 作为默认的执行方式

- eager 模式目前只支持单 GPU 运行，在 0.5.0 会支持多 GPU 运行


其他更新

新的 Python Pip 包名和版本号规则
之前一个 OneFlow 的版本采取的是“不同包名，相同版本名”的规则，如 `oneflow_cu102==0.3.4`，从 0.4.0 之后将采取“相同包名，不同版本名”的规则，如`oneflow==0.4.0+cu102`，最新安装方式请参考 README [Install with Pip Package章节](https://github.com/Oneflow-Inc/oneflow#install-with-pip-package)