PyPI v1.17.0 - ClearML

New Features

• Add programmatic pipeline clone using PipelineController.create() and PipelineController.clone() (#1353)
• Add Python 3.13 support
• Add support for local imports in pipeline steps
• Add support for the ClearML HTTP router using Task.get_http_router()
• Add TCP protocol support to Task.request_external_endpoint()
• Update pyjwt version

Bug Fixes

• Fix slow handling of cached files with large cache_file_limit (#1352)
• Fix pipeline crash when repository is set to a directory
• Fix token is not renewed when using an external token (CLEATML_AUTH_TOKEN) and no credentials
• Fix don't download external files from parent datasets if they have been modified/removed in the child dataset

PyPI v1.16.5 - ClearML

New Features

• Add sdk.development.artifacts.auto_pickle configuration option to support changing the default pickle behavior when uploading artifacts
• Add silent_on_errors argument to Task.delete_artifacts() (default False)
• Add support for last change time in triggers using tags
• Add Task.request_external_endpoint() to request external endpoints on supported backends

Bug Fixes

• Fix clearml-data search CLI error if dataset version is None (#1329, thanks @d-vignesh!)
• Fix maxfile attribute dropped in psutil v6.0.0 causing an error to be printed
• Fix api.auth.req_token_expiration_sec configuration option to api.auth.request_token_expiration_sec (matches agent setting, keep backwards compatibility)
• Bump six version due to Python 3.12 issue with six.moves
• Fix bar charts with only 1 bar are not reported correctly

PyPI v1.16.4 - ClearML

New Features

• Add custom task binary support to clearml-task and CreateAndPopulate (allows bash script execution, requires agent version >=1.9.0)
• Add support for a default extension name when uploading a pandas dataframe artifact (see sdk.development.artifacts.default_pandas_dataframe_extension_name configuration option)
• Add verify field support for downloadable URL instead of a file path (see sdk.aws.s3 configuration section)

Bug Fixes

• Fix valid model URL might be overridden by an invalid one in case the upload failed

PyPI v1.16.3 - ClearML

New Features

• Add --tags option to clearml-task (#1284)
• Add retries parameter to StorageManager.upload_folder() (#1305)
• Add clearml-task and CreateAndPopulate support for bash scripts, ipynb and python modules (requires clearml-agent v1.9+)
• Add support for HTTP file upload progress reporting
• Add CLEARML_MULTI_NODE_SINGLE_TASK (values -1, 0, 1, 2) for easier multi-node single Task workloads
• Add Model.original_task property to models
• Change Model.task property to return connected task
• Update docstring on allowing users to pass packages=False to revert to requirements.txt inside their git repository

Bug Fixes

• Fix Kerastuner framework and examples (#1279)
• Fix scalar logging bug with Fire (#1301, thanks @tvelovraf!)
• Fix support passing folder to Task.get_script_info() to get the git info
• Fix Task.launch_multi_node() to enforce the parent of sub-tasks to be the master node 0 task
• Fix tensorboard numpy 2.0 incompatibility breaks binding
• Fix Task.launch_multi_node() not supported when used via Pytorch Lightning
• Fix Jupyter notebook packages and uncommitted changes are sometimes not fetched
• Fix "can't create new thread at interpreter shutdown" errors (known issue with Python v3.12.0 and other versions)
• Fix injected task import in Task.populate()
• Fix dataset with external links will not reuse downloaded data from parents
• Fix hierarchy for pipeline nodes without args
• Fix when abort callback is set, set task status to stopped only if running locally, otherwise leave it for the Agent to set it
• Fix jsonschema / referencing import to include TypeError protection
• Fix Dataset offline behavior

PyPI v1.16.2 - ClearML

New features

• Make dataset preview optional (#1270, thanks @pedroconceicao!)
• Add api.public_ip_ping (default: 8.8.8.8) and api.public_ip_service_urls (default: ["api.ipify.org";, "ident.me";]) configuration settings to detect public IP and network interface
• Improve Dataset.get_mutable_local_copy() docstring

Bug Fixes

• Fix typo in docs and default SDK config (#1281, thanks @jimdiroffii!)
• Fix python-fire integration (#1275, thanks @tvelovraf!)
• Fix path substitution for file:// URIs (#1251, thanks @nfzd!)
• Fix numpy 2.0 compatibility (np.NINF removed)
• Fix no need to recreate reporter if forking and reporting in subprocess
• Fix forked detection mechanism

PyPI v1.16.1 - ClearML

Bug Fixes

• Fix pipeline breaks when continue_on_abort is set to true
• Fix Pycharm Plugin Windows/Linux interoperability

PyPI v1.16.0 - ClearML

New Features

• Add additional warning instructing on how to install in case we failed detecting a Jupyter notebook with an import error
• Add Task.get_executed_queue() to get the name/ID of the queue a task was executed in
• Move Task.set_resource_monitor_iteration_timeout() to a class method, add wait_for_first_iteration_to_start_sec and max_wait_for_first_iteration_to_start_sec arguments (also add sdk.development.worker.wait_for_first_iteration_to_start_sec and sdk.development.worker.max_wait_for_first_iteration_to_start_sec configuration options)
• Add support for better pipeline continue behavior including control of children using the continue_behaviour argument
• Add Python 3.12 support

Bug Fixes

• Fix issue #1249 pytorch-lightning patches (#1254, thanks @a-gardner1!)
• Fix parameter overrides are converted to string when using HPO (#975)
• Fix FastAI performance (#1234)
• Fix MIG GPU support
• Fix AMD GPU metrics collection
• Fix Jupyter password might not be used in some protected Jupyterlab instances
• Fix URL substitution was not applied to registered uploaded files when reporting an event

PyPI v1.15.1 - ClearML

Bug Fixes

• Fix auto-scaler should recheck that the worker is still IDLE before shutting it down (#1240, thanks @cthorey!)
• Fix resource monitor fails to get GPU stats in some edge cases

PyPI v1.15.0 - ClearML

New Features

• Add draft option to pipeline steps (#1226, thanks @CharlesFrankum!)
• Add support for custom working directory for pipelines (#1194)
• Add Task.get_requirements() method returning the task’s requirements
• Allow controlling the number of threads used by StorageManager.download_folder() using the max_workers argument
• Update examples dependencies
• Improve auto populate in Task.init()
• Documentation:
  • Add docstrings for model properties
  • Improve docstring for force_requirements_env_freeze
  • Add Task.set_packages() docstring notes

Bug Fixes

• Fix UTF-8 script code encoding issue (#1208, thanks @ae-ae!)
• Fix Colab docs (#1220, thanks @tkukurin!)
• Fix metrics reporting with OutputModel while in offline mode results in an error (#1172)
• Fix task running in Google Colab doesn't properly get the notebook diff (#1204)
• Fix hydra binds break in offline mode (#1215)
• Fix HPO crashes when optimizing for single value scalars (#1221)
• Fix GPU info such as gpu_memory and gpu_type is not being collected in some cases
• Fix clearml-data CLI tool will move non-dataset tasks to a .dataset project if the respective task is not a dataset
• Fix pandas DataFrame artifacts with hierarchical indices get mangled by the CSV round-trip
• Fix urllib3 sends deprecation warning when setting ssl_version
• Fix Task.connect(dict) return value is not dict-compatible
• Fix jsonargparse sub-command config parsing
• Fix Lightning integration crashes when a config entry contains . in its name
• Fix Python 3.5 compatibility

PyPI v1.14.4 - ClearML

Bug Fixes

• Fix AWS S3 configuration secure property not forwarded in StorageHelper (#1205, thanks @odulcy-mindee!)
• Fix NVIDIA_VISIBLE_DEVICES=all env var value will result in no GPU metrics being reported
• Fix setting output_uri when an AWS S3 config section with secure=false exists raises an error
• Fix CacheManager attempts to clear cache when file limit has not been reached resulting in slow unnecessary checks