Skip to content

Commit

Permalink
update with third review
Browse files Browse the repository at this point in the history
  • Loading branch information
@stonezdj
stonezdj committed Feb 28, 2024
1 parent be5b96f commit 1bebac5
Showing 1 changed file with 35 additions and 32 deletions.
67 changes: 35 additions & 32 deletions proposals/new/sbom_gen_scan.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,9 @@ Scan SBOM

- Generate SBOM for artifacts by 3rd tools and stores the SBOM as accessory in Harbor.
- No limitation of the SBOM generation tool, provide the flexible ways to attach the SBOM.
- **Scanner Mode**: Extend the spec of [pluggable scanner](<https://github.com/goharbor/pluggable-scanner-spec>). The scanner provides additional SBOM generation and scanning capabilities. Harbor is responsible for sending SBOM generation or scanning requests to the scanner. When a request is received, the scanner retrieves the subject artifact and generates the SBOM. Additionally, when Harbor sends a scan request for the SBOM, the scanner scans for vulnerabilities within it.
- The SBOM will be listed as the accessory of the artifact in the harbor.
- Other interact functions based on different scenarios after integrating SBOM to harbor.
- **Scanner Mode**: Extend the spec of [pluggable scanner](<https://github.com/goharbor/pluggable-scanner-spec>). The scanner provides additional SBOM generation and scanning capabilities. Harbor is responsible for sending SBOM generation or scanning requests to the scanner. When a request is received, the scanner retrieves the subject artifact and generates the SBOM.
- The SBOM will be listed as the accessory of the artifact in the Harbor.
- The SBOM can be downloaded from the Harbor UI.


## Personas
Expand All @@ -69,7 +69,7 @@ Scan SBOM
1. As the system administrator, I want to add a scanner to Harbor which has the ability of generate and scan SBOM.
2. As the project administrator, I want to add a custom scanner instead of system default one to Harbor which has the ability of generate and scan SBOM.
3. As the system/project administrator, I want to enable the SBOM generation automatically for a project.
4. As the project maintainer, I want to trigger the generate or scan operation for the SBOM manually.
4. As the project maintainer, I want to trigger the generate SBOM operation manually.
5. As the authorized user with read permission, I want to view the SBOM accessories if I can see the sbom information. and can download it to local.
6. As the authorized user with delete permission, I want to delete the SBOM accessories if I can delete the subject artifact.

Expand All @@ -94,6 +94,11 @@ ALTER TABLE scanner_registration ADD COLUMN IF NOT EXISTS spec_version varchar(5
```

The scan_report table should be updated to add the following columns
```
ALTER TABLE scan_report ADD COLUMN IF NOT EXISTS media_type varchar(255);
```

### Adatper API change
The adapter need to implement the following APIs to support the SBOM generation and scan.

Expand Down Expand Up @@ -286,9 +291,9 @@ If the sbom_media_type is not provided, the scanner adapter should return 400 to

1. Add a project level configuration item auto_gen_sbom to enable auto generate SBOM on image push, the default value is false.

2. Add a system configuration item to set default SBOM media_type for scanner, its value could be `application/spdx+json` or `application/vnd.cyclonedx+json`
2. The default SBOM media_type for scanner is hardcode to `application/spdx+json`, the media_type is stored in scan_report table as a column.

3. Update scanners API to add two attribute for scanners. the attribute is support_vulnerability, support_sbom, they are calculated value from scanner capabilities. if the scanner is spec 1.1 or older, the support_vulnerability should be true, and support_sbom should be false.
3. Update scanners API to add two attributes for scanners. the attribute is support_vulnerability, support_sbom, they are calculated value from scanner capabilities. if the scanner is spec 1.1 or older, the support_vulnerability should be true, and support_sbom should be false.
for the default trivy scanner, the support_vulnerability and support_sbom should be true.
```
GET /api/v2.0/scanners
Expand Down Expand Up @@ -332,7 +337,7 @@ Request Body:
}
```

When the scan_type is empty or `vulnerability`, the scan type should be vulnerability scan, if the scan_type is `sbom`, the scan type should be SBOM generation. The SBOM generation job will be created and executed asynchronously. it will use the default scanner to scan the artifact. if the scanner is not available, the job will be failed.
When the scan_type is empty or `vulnerability`, the scan type should be vulnerability scan, if the scan_type is `sbom`, the scan type should be SBOM generation. The scan job will be created and executed asynchronously. it will use the default scanner to scan the artifact. if the scanner is not available, the job will be failed.

If the scan request is accepted, it will return 202 to the client, request body is empty.

Expand All @@ -348,18 +353,25 @@ Request Body:
"scan_type":"sbom"
}
```
If the scan_type is empty, it is default to "vulnerability". it will query the execution table to find the scan job with the same artifact digest and the scan type. if the job is running, it will stop the job. The execution's extra_attrs should be updated to include the scan_type information.
If the scan_type is empty, it is default to "vulnerability". it will query the execution table to find the scan job with the same artifact digest and the scan type. if the job is running, it will stop the job.

When the scan_type is empty or `vulnerability`, the scan type should be vulnerability scan, if the scan_type is `sbom`, the scan type should be SBOM generation. The SBOM generation job can be stopped. if the scanner is not available, the job will be failed.

6. Update existing scan job service, add support to generate SBOM for an artifact, update the existing job service in pkg/scan/job.go to support SBOM generate in the request parameter. for a generate SBOM report request, the job service should:

6. Update existing scan job service, add support to generate SBOM for an artifact, the existing job service in pkg/scan/job.go to support SBOM generate in the request parameter. the previous vulnerability scan job is still supported. after receiving the SBOM report, the job service should:

1. Delete the previous SBOM report generated with the same scanner if exist. include the information in the table scan_report and the artifact accessory in the OCI registry. if current scan type is SBOM, it will get the `sbom_digest` and `sbom_repository` information from the scan_report table's report column, if both exist, then it will delete the SBOM accessory in the OCI registry.
1. Delete the previous SBOM report generated with the same scanner if exist. include the information in the table scan_report and the artifact accessory in the OCI registry. if current scan type is SBOM, it will get the `sbom_digest` and `sbom_repository` information from the scan_report table's report column, if both exist, then it will delete the SBOM accessory in the OCI registry.
1. Parse the SBOM report and push the the SBOM to the OCI registry as an artifact accessory. its media type should be "application/vnd.goharbor.harbor.sbom.v1"
1. Store the SBOM digest information into the table scan_report.

The generic process of the SBOM generation and scan is as following:
The Harbor job service stores the SBOM digest information into the database. the scan_report table's report column is updated with the following content when it is a sbom report:
```
{
"sbom_digest":"sha256:51a41cec9de9d62ee60e206f5a8a615a028a65653e45539990867417cb486285",
"sbom_repository":"library/nginx"
}
```
If there is a SBOM report exists for the same scanner, it should be replaced.
The generic process of the SBOM generation and scan is as following:
1. Parse scan request job parameters
2. Select the scanner to scan the artifact
Expand All @@ -368,7 +380,7 @@ When the scan_type is empty or `vulnerability`, the scan type should be vulnerab
5. The job service query the scan report from the scanner adapter by the given request id, until all required reports are ready. it also have a timeout to avoid the job to be blocked too long.
6. Parse the report and store the report information.
7. Update existing list artifact API to support list artifact with_sbom_overview, if with_sbom_overview is true, the sbom_overview data should be provided.
7. Update existing list artifact API to support list artifact with_sbom_overview, if with_sbom_overview is true and the sbom information is in the scan_report, the sbom_overview data should be provided.
```
[
{
Expand All @@ -390,6 +402,7 @@ When the scan_type is empty or `vulnerability`, the scan type should be vulnerab
"report_id": "1a6f49a5-17ea-49b4-94ff-f38fc80cc0c8",
"scan_status": "Stopped",
"start_time": "2024-01-30T08:07:52.000Z"
"sbom_digest": "sha256:51a41cec9de9d62ee60e206f5a8a615a028a65653e45539990867417cb486285",
}
},
...
Expand All @@ -399,7 +412,7 @@ When the scan_type is empty or `vulnerability`, the scan type should be vulnerab
}
]
```
8. Add an API to retrieve the SBOM report for the artifact
8. Add an API to retrieve the SBOM content for the SBOM accessory.
Request Method:
```
GET /api/v2.0/projects/<projectname>/repositories/<repository>/artifacts/<digest>/additions/sbom
Expand Down Expand Up @@ -437,22 +450,13 @@ Response:
}
```
The additions API should be updated to support the SBOM report. the artifact digest is the sbom accessory digest and the actual sbom content is stored in the OCI registry as a layer, it requires to register a sbom processor for the media type "application/vnd.goharbor.harbor.sbom.v1". the processor should implement the method of AbstractAddition, in this method it parse the first layer of the artifact accessory, and return the SBOM content in the response body.
The additions API should be updated to support the SBOM report. the artifact digest is the SBOM accessory digest and the actual sbom content is stored in the OCI registry as a layer, it requires to register a sbom processor for the media type "application/vnd.goharbor.harbor.sbom.v1". the processor should implement the method of AbstractAddition(), in this method it parse the first layer of the artifact accessory, and return the SBOM content in the response body.
Because the end user could upload the SBOM to an artifact, and also Harbor itself could generate SBOM for an existing artifact. When all projects enabled auto generated sbom on push, the number of SBOMs is approximately equal to the number of artifacts. for performance consideration, the SBOM should be stored in the OCI registry, not in the postgres database. Harbor UI only displays the SBOM report generated in Harbor.
If the SBOM report is unavailable, it returns http code 404 to the client.
Given a artifact digest, if the SBOM report is unavailable, it returns http code 404 to the client.
The Harbor job service stores the SBOM digest information into the database. the scan_report table's report column is updated with the following content when it is a sbom report:
```
{
"sbom_digest":"sha256:51a41cec9de9d62ee60e206f5a8a615a028a65653e45539990867417cb486285",
"sbom_repository":"library/nginx"
}
```
If there is a SBOM report exists for the same scanner, it should be replaced.

9. In the list artifact page, add UI tab to display the SBOM detail information of the artifact, it retrieves the scan_report table and get the digest of the artifact accessory, if the artifact accessory exists, it returns the SBOM status information. If the report in scan_report is empty, the SBOM status keeps empty.
9. In the list artifact page, add SBOM summary column to display the SBOM summary information of the artifact, it retrieves the scan_report table and get the digest of the artifact accessory, if the artifact accessory exists, it returns the SBOM status information. If the report in scan_report is empty, the SBOM summary keeps empty or fallback to the query result of the artifact accessory for this artifact.
10. In the artifact detail information page artifact accessory, add UI tab to display the SBOM information of the SBOM accessory, it only supports the SBOM generated by Harbor.
Expand All @@ -471,24 +475,23 @@ If there is a SBOM report exists for the same scanner, it should be replaced.
The SBOM is stored the OCI registry, it should:
1. It can be replicated by Harbor replication
1. It can be replicated by Harbor replication, except missing SBOM summary information, the target Harbor should display the same SBOM detail information. when a SBOM artifact accessory is replicated from Harbor A to Harbor B, then list the artifact in Harbor B, the SBOM accessory should be displayed in the artifact's list page. because the scan_report in Harbor B has no sbom report associate with the artifact. then it will fallback to the query result of artifact accessory for this artifact, if there is a SBOM artifact accessory, then the artifact's SBOM status column will display the link to the SBOM detail page. it is handled by UI logic.
2. If the artifact is delete, the SBOM information should be removed as well
3. If it is generated by the current scanner in the Harbor, it should removed when a new SBOM generation start. for each artifact, it only keeps the latest SBOM report generated by the current scanner.
3. If a SBOM accessory is deleted, Harbor should remove it from scan_report table by digest and repository information stored in the report column. this logic should be implemented in the internal artifact event hook.
4. If it is generated by the current scanner in the Harbor, it should removed when a new SBOM generation start. for each artifact, it only keeps the latest SBOM report generated by the current scanner.
### UI
1. Configuration item to enable auto generate SBOM on push. if the current scanner doesn't support to generate SBOM, the configuration item should be disabled.
![Generate sbom on push](../images/sbom/config_gen_sbom.png "Generate sbom on push")
2. System configuration item to select the default SBOM media_type, it could be `application/spdx+json` or `application/vnd.cyclonedx+json`

2. Add button to generate SBOM in the artifact table, manually generate SBOM for the artifact. the button should be disabled if the current scanner does not support to generate SBOM. the generate SBOM button and stop generate SBOM button should be mutex, the stop generate SBOM and stop scan should have the same backend API with different parameters.
![Generate sbom](../images/sbom/generate_sbom_btn.png "Generate sbom")
3. In the artifact table, if there is sbom_enabled, it will display the SBOM status. the SBOM status could link to the SBOM detail information in #4.
![Display sbom](../images/sbom/display_sbom.png "Display sbom")
4. Display the full content of the current SBOM (include spdx and cyclonedx). the content of the SBOM can be download as a file.
4. Display the full content of the current SBOM. the content of the SBOM can be download as a file.
![Display sbom](../images/sbom/display_sbom_detail.png "Display sbom detail")
Expand Down

0 comments on commit 1bebac5

Please sign in to comment.