PyPI v1.2.0 - ClearML

Features

• Add fastai v2 support (#571)
• Add catboost support (#542)
• Add Python Fire support (#550)
• Add new Azure Storage driver support (#548)
• Add requirements file support in Task.add_requirements (#575)
• Allow overriding auto_delete_file in Task.update_output_model() (#554)
• Support artifact_object empty string
• Add skip_zero_size_check to StorageManager.download_folder()
• Add support for extra HTTP retry codes (see here or use CLEARML_API_EXTRA_RETRY_CODES)
• Add Task.get_parameters() cast back to original type
• Add callback support to Task.delete()
• Add autoscaler CPU-only support
• Add AWS autoscaler IAM instance profile support
• Update examples
  • Edit HTML reporting examples (#546)
  • Add model reporting examples (#553)

Bug Fixes

• Fix nargs="?" without type does not properly cast the default value (#531)
• Fix using invalid configurations (#544)
• Fix extra_layout not passed to report_matrix (#559)
• Fix group arguments in click (#561)
• Fix no warning when failing to patch argparse (#576)
• Fix crash in Dataset.upload() when there is nothing to upload (#579)
• Fix requirements, refactor and reformat examples (#567, #573, #582)
• Auto-scaler
  • Change confusing log message
  • Fix AWS tags support
  • Fix instance startup script fails on any command (should only fail on the agent failing to launch)
  • Fix spin down stuck machine, ignore unknown stale workers
• Fix pandas object passed as Task.upload_artifact() preview object
• Fix incorrect timeout used for stale workers
• Fix clearml-task calls Task.init() in the wrong place when a single local file is used
• Fix ArgumentParser SUPPRESS as default should be resolved at remote execution in the same way (i.e. empty string equals SUPPRESS)
• Upgrade six version (in case pathlib2>2.3.7 is installed)
• Fix connected object base class members are not used
• Fix clearml-init changing web host after pasting full credentials
• Fix fileserver upload does not support path in URL
• Fix crash on semaphore acquire error
• Fix docs and docstrings (#558, #560)

Thanks @eugen-ajechiloae-clearml, @pollfly and @Rizwan-Hasan for contributing!

PyPI v1.1.6 - ClearML

Features

• Add Task.force_store_standalone_script() to force storing standalone script instead of a Git repository reference (#340)
• Add Logger.set_default_debug_sample_history() and Logger.get_default_debug_sample_history() to allow controlling maximum debug samples programmatically
• Add populate now stores function arg types as part of the hyper paremeters
• Add status_message argument to Task.mark_stopped()
• Change HTTP driver timeout and retry codes (connection timeout will now trigger a retry)

Bug Fixes

• Fix and upgrade the SlackMonitor (#533)
• Fix network issues causing Task to stop on status change when no status change has occurred (#535)
• Fix Pipeline controller function support for dict as input argument
• Fix uploading the same metric/variant from multiple processes in threading mode should create a unique file per process (since global counter is not passed between the subprocesses)
• Fix resource monitoring should only run in the main process when using threaded logging mode
• Fix fork patching so that the signal handler (at_exit) will be called on time
• Fix fork (process pool) hangs or drops reports when reports are at the end of the forked function in both threaded and subprocess mode reporting
• Fix Multi-pipeline support
• Fix delete artifacts after upload
• Fix artifact preview has no truth value
• Fix storage cache cleanup does not remove all entries on a silent fail
• Fix always store session cache in ~/.clearml (regardless of the cache folder)
• Fix StorageManager.download_folder() fails on Windows path

PyPI v1.1.5 - ClearML

Features

• Add support for jsonargpraser (#403)
• Add HyperParameterOptimizer.get_top_experiments_details() returns the hparams and metrics of the top performing experiments of an HPO (#473)
• Allow overriding initial iteration offset using environment variable (CLEARML_SET_ITERATION_OFFSET) or Task.init(continue_last_task==<offset>) (#496)
• Add better input handling for clearml-init in colab (#515)
• Add environment variable for default request method (#521)
• Add LocalClearmlJob as possible option for HPO (#525)
• Add convenience functionality to clearml-data (#526)
• Add support for vscode-jupyter (microsoft/vscode-jupyter#8531)
• Improve detection of running reporting subprocess (including zombie state)
• Support controlling S3/Google Cloud Storage _stream_download_pool_connections using the stream_connections configuration setting in clearml.conf (default 128)
• Add warning when loosing reporting subprocess
• Add Model.remove() to allow removing a model from the model repository
• Add HTTP download timeout control (change default connection timeout to 30 seconds)
• Add initial setup callback to monitoring class
• Add Task.get_reported_plots()
• Allow Monitor.get_query_parameters to override defaults
• Add support for Google Cloud Storage pool_connections and pool_maxsize overrides
• Add last worker time to AutoScaler
• Add warning when opening an aborted Dataset
• Store multi-pipeline execution plots on the master pipeline Task
• Support pipeline return value stored on pipeline Task
• Add PipelineDecorator.multi_instance_support
• Add PipelineDecorator to clearml and clearml.automation namespaces
• Documentation and examples
  • Update docstrings (#501)
  • Add Markdown in pipeline jupyter notebooks (#502)
  • Update pipeline example (#494)
  • Add abseil example (#509)
  • Change README to dark theme (#513)
  • Update XGBoost example (#524)
  • Change example name (#528)

Bug Fixes

• Fix TriggerScheduler on Dataset change (#491)
• Fix links in Jupyter Notebooks (#505)
• Fix pandas delta datetime conversion (#510)
• Fix matplotlib auto-magic detect bar graph series name (#518)
• Fix path limitation on storage services (posix, object storage) when storing target artifacts by limiting length of project name (full path) and task name used for object path (#516)
• Fix multi-processing context block catching exception
• Fix Google Cloud Storage with no default project causes a crash
• Fix main process's reporting subprocess lost, switch back to thread mode
• Fix forked StorageHelper should use its own ThreadExecuter
• Fix local StorageHelper.delete() raising exception on non existing file instead of returning false
• Fix StorageHelper rename partial file throwing errors on multiple access
• Fix resource monitor fails on permission issues (skip over parts)
• Fix reusing Task does not reset it
• Fix support clearml PyCharm Plugin 1.0.2 (support partial pycharm git repo sync)
• Fix Task.reset() force argument ineffective
• Fix PY3.5 compatibility
• Fix validation error causes infinite loop
• Fix tasks schema prevents sending null container parts
• Fix missing CLEARML_SET_ITERATION_OFFSET definition
• Fix Model.get_weights_package() returns None on error
• Fix download progress bar based on sdk.storage.log.report_download_chunk_size_mb configuration
• Fix Conda lists the CudaToolkit version installed (for the agent to reproduce)
• Fix Jupyter kernel shutdown causing nested atexit callbacks leaving Task in running state
• Fix multi-subprocess can cause Task to hand at close
• Fix TF 2.7 support (get logdir on with multiple TB writers)

PyPI v1.1.4 - ClearML

Bug Fixes

• Fix duplicate keyword argument (affects clearml-data Dataset.get()) #490, ClearML Slack Channel #1, #2, #3, #4
• Fix session raises missing host error when in offline mode #489
• Fix Task.get_task() does not load output_uri from stored Task
• Fix Task.get_models()['input'] returns string instead of clearml.Model
• Fix tf.saved_model.load() binding for TensorFlow >=2.0
• Fix hyperparams with None value converted to empty string causes inferred type to change to str in consecutive Task.connect() calls

PyPI v1.1.3 - ClearML

Features

• Add support for MegEngine with examples (#455)
• Add TaskTypes to main namespace (#453)
• Add LogUnifomParameterRange for hyperparameter optimization with Optuna (#462)
• Add joblib (equivalent to scikit) to Task.init(auto_connect_frameworks) argument
• Log environment variables starting with * in environ_bind.py (#459)
• Pipeline
  • Add eager decorated pipeline execution
  • Support pipeline monitoring for scalers/models/artifacts
  • Add PipelineController.upload_model()
  • Add PipelineController.add_step(configuration_overrides) argument allowing to override Task configuration objects
  • Change PipelineController.start_locally() default run_pipeline_steps_locally=False
  • Add PipelineController.stop(mark_failed, mark_aborted) argguments
  • Add PipelineController.run_locally decorator
  • Add PipelineController.continue_on_fail property
  • Add PipelineController.__init__(abort_on_failure) argument
  • Add ClreamlJob state cache (refresh every second)
• Datasets
  • Add clearml-data multi-chunk support
  • Change clearml-data default chunk size to 512MB
  • Change Dataset.create() now automatically reverts to using current Task if no project/name provided
• Add Optimizer.get_top_experiments_id_metrics_pair() for top performing experiments
• Add support for setting default value to auto connected argparse arguments
• Add Task.get_script() and Task.set_script() for getting and setting task's script prioerties for execution
• Add Task.mark_completed() force and status_message arguments
• Add Task.stopped() reason argument
• Add Task.query_tasks(), Task.get_task() and Task.get_tasks() tags argument

Bug Fixes

• Fix PyJWT resiliency support
• Fix xgb train overload (#456)
• Fix http:// throws OSError in Windows by using pathlib2 instead of os (#463)
• Fix local diff should include staged commits, otherwise applying git diff fails (#457)
• Fix task.upload_artifact non-standard dictionary will now revert to pickle (#452)
• Fix S3BucketConfig.is_valid() for EC2 environments with use_credentials_chain (#478)
• Fix audio classifier example when training with a custom dataset (#484)
• Fix clearml-task diff was corrupted by Windows drive letter and separator (#483)
• Fix TQDM "line cleanup" not using CR but rather arrow-up escape sequence (#181)
• Fix task.connect(dict) value casting - if None is the default value, use backend stored type
• Fix Jupyter notebook should always set Task as completed/stopped, never failed (exceptions are caught in interactive session)
• Fix Pipeline support
  • Fix LocalClearmlJob setting failed status
  • Fix pipeline stopping all running steps
  • Fix nested pipeline component parent point to pipeline Task
  • Fix PipelineController.start() should not kill the process when done
  • Fix pipeline failing to create Step Task should cause the pipeline to be marked failed
  • Fix nested pipeline components missing pipeline tags
• Fix images reported over history size were not sent if frequency was too high
• Fix git detectors missing git repository without origin
• Fix support for upload LazyEvalWrapper artifacts
• Fix duplicate task dataset tags
• Fix FileLock create target folder
• Fix crash inside forked subprocess might leave SafeQueue in a locked state, causing task.close() to hang
• Fix PyTorch distributed example TimeoutSocket issue in Windows
• Fix broken Dataset.finalize()
• Fix Python 3.5 compatibility

PyPI v1.1.2 - ClearML

Bug Fixes

• Fix PyJWT issue (limit dependency to <2.2.0)

PyPI v1.1.1 - ClearML

Bug Fixes

• Fix Logger.report_image() throws warning
• Fix TensorBoard add_image() not being reported

PyPI v1.1.0 - ClearML

Breaking Changes

• New PipelineController v2 (note: new constructor is not backwards compatible)
• Disable default demo server (available by setting the CLEARML_NO_DEFAULT_SERVER=0 environment variable)
• Deprecate Task.completed() (use Task.mark_completed() instead)

Features

• Add Task Trigger Scheduler
• Add Task Cron Scheduler
• Add PipelineController from function
• Add PipelineDecorator (PipelineDecorator.pipeline and PipelineDecorator.component decorators for full custom pipeline logic)
• Add xgboost auto metric logging #381
• Add sdk.storage.log.report_upload_chunk_size_mb and sdk.storage.log.report_download_chunk_size_mb configuration options to control upload/download log reporting #424
• Add new optional auto_connect_frameworks argument value to Task.init() (e.g. auto_connect_frameworks={'tfdefines':False}) to allow disabling TF defines #408
• Add support for CLEARNL_CONFIG_VERBOSE environment variable to allow external control over verbosity of the configuration loading process
• Add support for uploading artifacts with a list of files using Task.upload_artifcats(name, [Path(), Path()])
• Add missing clearml-task parameters --docker_args, --docker_bash_setup_script and --output-uri
• Change CreateAndPopulate will auto list packages imported but not installed locally
• Add clearml.task.populate.create_task_from_function() to create a Task from a function, wrapping function input arguments into hyper-parameter section as kwargs and storing function results as named artifacts
• Add support for Task serialization (e.g. for pickle)
• Add Task.get_configuration_object_as_dict()
• Add docker_image argument to Task.set_base_docker() (deprecate docker_cmd)
• Add auto_version_bump argument to PipelineController
• Add sdk.development.detailed_import_report configuration option to provide a detailed report of all python package imports
• Set current Task as Dataset parent when creating dataset
• Add support for deferred configuration
• Examples
  • Add Pipeline v2 examples
  • Add TaskScheduler and TriggerScheduler examples
  • Add pipeline controller callback example
  • Improve existing examples and docstrings

Bug Fixes

• Fix poltly plots converting NaN to nan instead of null #373
• Fix deprecation warning #376
• Fix plotly multi-index without index names #399
• Fix click support #437
• Fix docstring #438
• Fix passing task-type to clearml-task #422
• Fix clearml-task --version throws an error #422
• Fix clearml-task ssh repository links are not detected as remote repositories #423
• Fix getattr throws an exception #426
• Fix encoding while saving notebook preview #443
• Fix poetry toml file without requirements.txt #444
• Fix PY3.x fails calling SemLock._after_fork with forkserver context, forking while lock is acquired allegroai/clearml-agent#73
• Fix wrong download path in StorageManager.download_folder()
• Fix jupyter notebook display(...) convert to print(...)
• Fix Tensorflow add_image() with description='text'
• Fix Task.close() should remove current_task() reference
• Fix TaskScheduler weekdays, change default execute_immediately to False
• Fix Python2 compatibility
• Fix clearml-task exit with error when failing to verify output_uri (output warning instead)
• Fix unsafe Google Storage delete object
• Fix multi-process spawning wait-for-uploads can create a deadlock in very rare cases
• Fix task.set_parent() fails when passing Task object
• Fix PipelineController skipping queued Tasks
• Remove humanfriendly dependency (unused)

PyPI v1.0.5 - ClearML

Features

• Add Click support and examples #386
• Add progress bar to SHA2 generation #396
• Add prefix to Task reported runtime info: cpu_cores, gpu_driver_version and gpu_driver_cuda_version
• Add support for Logger.report_text() explicit log-level reporting
• Add return_full_path argument to StorageManager.list()
• Support Task.get_tasks() passing multiple project names
• Add TaskScheduler
• Add task_filter argument to Objective.get_top_tasks(), allow name as a task_filter field
• Add --output-uri command-line option to clearml-task
• Add requirements_file argument to Task.force_requirements_env_freeze() to allow specifying a local requirements file
• Add support for list type argument in Task.connect_configuration() (previously only dict type was supported)
• Rename TrainsTuner to ClearmlTuner
• Update documentation links

Bug Fixes

• Fix Pandas with multi-index #399
• Fix check permissions fail in HTTPDriver #394
• Fix Dataset not setting system tag on existing data_processing Tasks
• Fix disable redundant resource monitoring in pipeline controller
• Fix ClearMLJob when both project and target_project are specified
• Fix ClearMLJob docker container info is not cached
• Fix no print logging after Python logging handlers are cleared
• Fix PipelineController callback returning False
• Fix machine specs when GPU is not supported
• Fix internal logging.Logger can't be pickled (only applicable to Python 3.6 or lower)
• Wait for reported events to flush to ensure Task.flush() with wait_for_uploads=True awaits background processes

PyPI v1.0.4 - ClearML

Features

• Add Google Colab notebook tutorial #368 #374
• Add support for GIF images in Tensorboard #372
• Add a tensorboardX example for add_video (creates GIFs in tensorboard) #372
• Add auto scaler customizable boot bash script
• Add Task.ignore_requirements
• Deprecate Logger.tensorboard_single_series_per_graph() as it is now controlled from the UI 🙂

Bug Fixes

• Fix default_output_uri for Dataset creation #371
• Fix clearml-task failing without a docker script #378
• Fix Pytorch DDP sub-process spawn multi-process
• Fix Task.execute_remotely() on created Task (not initialized Task)
• Fix auto scaler custom bash script should be called last before starting agent
• Fix auto scaler spins too many instances at once then kills the idle ones (spin time is longer than poll time)
• Fix multi-process spawn context using ProcessFork kills sub-process before parent process ends