Home
Welcome to the reduction_ACES wiki!
This wiki provides an overview of finding specific data products for individual fields, spectral windows, molecular lines, mosaics, etc.
Using Globus to transfer ACES data
This wiki provides detailed instructions on how to get started, navigate the ACES data directories, set up personal endpoints, and utilize Globus for efficient data transfers. Whether you prefer using a web browser or the command line interface (CLI), you'll find step-by-step instructions and helpful resources to facilitate seamless transfers between the ACES-HPG endpoint and your personal endpoint.
Accessing ACES data using the Command Line Interface (CLI)
The provided instructions detail the basic setup and usage of Globus, a data transfer service, within the context of the ACES project. The process involves installing the necessary software, setting up personal endpoints, and activating them for data transfers. The instructions cover both the web browser interface and the command line interface (CLI) options for transferring data. Additionally, there are explanations on how to overcome permission issues and customize the writing permissions for endpoints. The script provided showcases an example of using the CLI to download a list of specific files from a source endpoint to a destination endpoint.
This page provides a high-level summary of various tasks related to data quality assurance (QA) in the context of the ACES project. Here is a breakdown of the different sections:
Getting started: This section provides an overview of how to begin the QA process. It includes instructions to access the ACES Execution Blocks (EBs) and identify the ones that are ready for QA. It also mentions the ideal number of people required for independent QA and provides instructions for assigning oneself to an EB.
Weblog review: The dataset processed by the ALMA pipeline generates a detailed weblog that is reviewed to identify potential issues. This section explains how to navigate the weblog and interpret the information provided. It covers checking warnings and QA scores for each task, examining specific examples of warnings, and understanding the significance of different types of warnings. The section also emphasizes the importance of reviewing pipeline images, including calibrator images, continuum results, full continuum images, and image cubes for each spectral window.
Manual image inspection: While the weblog provides informative images, it is recommended to download the full images for detailed inspection, especially when issues or warnings are identified. This section explains how to download pipeline image products using Globus and suggests specific directories to access the required image files.
Updating the GitHub issues: Each execution block has an associated GitHub issue where the QA summary is updated. This section instructs users on how to update the relevant issue by providing a summary of the weblog and image review. It also highlights the checklist present in each issue, where completed tasks can be checked and new tasks can be added based on identified issues or necessary recalibrations.
Manual reclean: In cases where re-imaging is required, this section provides guidance on using the CASA software for re-imaging and specifies the CASA version to use. It also mentions the specific directory structure to locate the tclean scripts and measurement sets needed for cleaning.
Updating clean parameter JSON files: When re-imaging a file, the clean commands used are preserved in the override_tclean_commands.json file. This section outlines common updates that may be needed in this file and provides instructions on how to submit changes as a pull request.
Uploading updated data: If a completed clean cube is imaged locally, this section explains how to upload it to Globus and provides instructions on where to note the uploaded file in the corresponding GitHub issue. It also mentions the process followed by UF personnel to rename, move, and symlink the uploaded files in the reclean/ directory for future reference.
Overall, this guide provides a step-by-step overview of the QA process for the ACES project, covering activities such as reviewing weblogs, inspecting images, updating GitHub issues, performing manual re-imaging, and uploading updated data.