Mrjob

Latest version: v0.7.4

Safety actively analyzes 681916 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 5 of 10

0.5.6

* Dataproc runner:
* fix Hadoop version crash on unknown image version (1428)
* EMR and Hadoop runners:
* prioritize task errors as probable cause of failure (1429)
* ignore Java stack trace in task stderr logs (1430)

0.5.5

* EMR runner:
* deprecate, don't remove ami_version option in v0.5.4 (1421)
* update memory/CPU stats for EC2 instances for pooling (1414)
* pooling treats application names as case-insensitive (1417)

0.5.4

* jobs:
* pass_through_option(), for existing command-line options (1075)
* MRJob.options.runner now defaults to None, not 'inline' or 'local'
* runners:
* all:
* names of uploaded files now never start with . or _ (1200)
* Hadoop:
* log parsing:
* handles more log4j patterns (1405)
* gracefully handles IOError from exists() (1355)
* fixed crash bug in Hadoop FS on Python 3 (1396)
* EMR:
* pooling auto-recovers from joining a cluster that self-terminated (708)
* log fetching uses sudo on 4.3.0+ AMIs (1244)
* fixed broken --ssh-bind-ports switch (1402)
* idle termination script now only runs on master node (1398)
* ssh tunnel connects to internal IP of resource manager (1397)
* AWS credentials no longer logged in verbose mode (1353)
* many option names are now more generic (1247)
* ami_version -> image_version
* accidentally removed ami_version option entirely (fixed in v0.5.5)
* aws_availability_zone -> zone
* aws_region -> region
* check_emr_status_every -> check_cluster_every
* ec2_core_instance_bid_price -> core_instance_bid_price
* ec2_core_instance_type -> core_instance_type
* ec2_instance_type -> instance_type
* ec2_master_instance_bid_price -> master_instance_bid_price
* ec2_master_instance_type -> master_instance_type
* ec2_task_instance_bid_price -> task_instance_bid_price
* ec2_task_instance_type -> task_instance_type
* emr_tags -> tags
* num_ec2_core_instances -> num_core_instances
* num_ec2_task_instances -> num_task_instances
* s3_log_uri -> cloud_log_dir
* s3_sync_wait_time -> cloud_fs_sync_secs
* s3_tmp_dir -> cloud_tmp_dir
* s3_upload_part_size -> cloud_upload_part_size
* num_ec2_instances is deprecated (use num_core_instances)
* ec2_slave_instance_type is deprecated (use core_instance_type)
* hadoop_streaming_jar_on_emr is deprecated (1405)
* hadoop_streaming_jar handles this instead with file:// URIs
* bootstrap_python does nothing on AMI 4.6.0+, as not needed (1358)
* mrjob audit-emr-usage should show less/no API throttling warnings (1091)

0.5.3

* jobs:
* LIBJARS and libjars method (1341)
* runners:
* all:
* .cpython-3*.pyc files no longer included when bootstrapping mrjob
* local:
*PATH envvars combined with local separator (1321)
* Hadoop and EMR:
* libjars option (198)
* fixes to ordering of generic and JAR-specific options (1331, 1332)
* Hadoop:
* more default log dirs (1339)
* hadoop_tmp_dir handles ~ and envvars (1322) (broken in v0.5.0)
* EMR:
* determine cause of failure of bootstrap scripts (370)
* master bootstrap script now redirects stdout to stderr
* emr_configurations option (1276)
* subnet option (1323)
* SSH tunnel opened as soon as cluster is ready (1115)
* SSH tunnel leaves stdin alone (1161)
* combine_lists() treats dicts as values, not sequences

0.5.2

* basic support for Google Cloud Dataproc (1243)
* lacks log interpretation, JarStep support
* on EMR, wait for steps to complete in correct order (1316)
* correctly handle ~ in include path in mrjob.conf (1308)
* new emr_applications option (1293)
* fix running deprecated tools with python -m (1312)
* fix ssh tunneling to 2.x AMIs on EMR in VPCs (1311)

0.5.1

* strict_protocols in mrjob.conf is no longer ignored (1302)
* check_input_paths in mrjob.conf is no longer ignored
* partitioner() is no longer ignored, fixing SORT_VALUES (1294)
* --partitioner switch is deprecated
* improved probable cause of error from pre-YARN logs (1288)
* ssh_bind_ports now defaults to (x)range, not list (1284)
* mrjob terminate-idle-clusters handles debugging jar from boto 2.40.0 (1306)

Page 5 of 10

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.