Skip to content

Commit

Permalink
Clean up comments
Browse files Browse the repository at this point in the history
  • Loading branch information
nicolemah99 committed Aug 13, 2024
1 parent dcd8619 commit fef57db
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 18 deletions.
7 changes: 2 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,20 +15,17 @@ This repository contains a set of Bash scripts that make up a data pipeline, des
## Prerequisites
Ensure you have the following before running the pipeline:
- Unix-like environment (Linux, macOS, or a Unix-like Windows terminal)
- Able to `ssh` to the **ot-dev** and **das.lab.acep.uaf.edu** servers
- FTP credentials for the meter
- Meter Configuration
- Installed on `camio-ot-dev`:
- Must have installed:
- `lftp`
- `yq`
- `zip`
- `rsync`
- `jq`

## Installation
1. You must be connected to the `camio-ot-dev` server. See **camio-ot-dev(SSH)** in the [ACEP Wiki](https://wiki.acep.uaf.edu/en/teams/data-ducts/aetb).

2. Clone the repository:
1. Clone the repository:

```bash
git clone [email protected]:acep-uaf/camio-meter-streams.git
Expand Down
2 changes: 1 addition & 1 deletion cli_meter/meters/sel735/generate_metadata_yml.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# meter_id Meter ID
# meter_type Meter type
# event_timestamp Original timestamp of the event
# started_at Timestamp of when the download started
# started_at Timestamp of when the download started
# completed_at Timestamp of when the download ended
#
# Requirements: common_utils.sh
Expand Down
15 changes: 3 additions & 12 deletions cli_meter/scripts/repackage-event-files.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,23 +7,14 @@
# naming convention.
#
# Naming Convention: location-data_type-meter-YYYYMM-event_id
# Ex. kea-events-meter1-202401-12345
#
# Example: Original zip file: 12345.zip
# Original message file: 12345.zip.message
#
# New zip file: kea-events-meter1-202401-12345.zip
# New message file: kea-events-meter1-202401-12345.zip.message
#
# Message File
# content: V1: id, filename, data_type, path
# if v1, repackaging is required
# V2: event_id, filename, path, data_type
# V3: event_id, filename, md5sum, data_type
#
# Usage: ./repackage-event-files.sh <BASE_DIR>
# example BASE_DIR: camio-meter-stream-test/data/kea/events/level0
# pwd /home/user/camio-meter-stream-test
#
# Arguments: BASE_DIR - The base directory to process
#
Expand Down Expand Up @@ -145,13 +136,13 @@ for date_dir in "$BASE_DIR"/*; do
echo "V1 Message Format: $message_file"
count_v1=$((count_v1 + 1))

# Parse the message file
# Parse message file
read event_id zip_filename data_type path<<< $(parse_message_file "$message_file")

# Parse location from path (first path) Ex: kea/path/to
# Parse location
location=$(echo "$path" | cut -d'/' -f1)

# New naming conventions example: kea-events-meter1-202401-12345
# New naming conventions: location-data_type-meter-YYYYMM-event_id
new_dir_name="${location}-${data_type}-${meter}-${fmt_date_dir}-${event_id}"
new_zip_file_name="${new_dir_name}.zip"
new_message_file_name="${new_zip_file_name}.message"
Expand Down

0 comments on commit fef57db

Please sign in to comment.