Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add check to not private builds to public channels #15041

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion eng/publishing/v3/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ stages:
/p:PublishingInfraVersion=3
/p:BARBuildId=${{ parameters.BARBuildId }}
/p:TargetChannels='${{ parameters.PromoteToChannelIds }}'
/p:IsInternalBuild=${{ contains(variables['AzDOBranch'], 'internal/') }}
/p:IsInternalBuild=$(IsInternalBuild)
/p:NugetPath=$(NuGetExeToolPath)
/p:MaestroApiEndpoint='$(MaestroApiEndPoint)'
/p:BuildAssetRegistryToken='$(AuthenticateMaestro.MaestroToken)'
Expand Down
6 changes: 6 additions & 0 deletions eng/publishing/v3/validate-and-locate-build.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,11 @@ try {
$buildNumberName = $buildNumberName.Substring(0, 255)
}

$isInternalBuild = $false
if ([string]::IsNullOrEmpty($buildInfo.gitHubRepository) -and $buildInfo.azureDevOpsBranch.Contains("internal/release")) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, this could attempt to anonymous access the commit in GH if the repo GH repo URI is available. That gets rid of the branch name check.

The downside of this is that you do need to implement safety check skipping if this happens. Some dev builds as well as non-public repos (wpf-int) that produce public assets would need to pass this switch.

Thoughts?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, this could attempt to anonymous access the commit in GH if the repo GH repo URI is available. That gets rid of the branch name check.
I don't mind this if you think it'd be a more reliable way of doing the check

The downside of this is that you do need to implement safety check skipping if this happens. Some dev builds as well as non-public repos (wpf-int) that produce public assets would need to pass this switch.

Yes, this is true for both cases. I think we'd have to:

  • Get a list of all default-channels that publish from non-public to public
  • Add a parameter to Darc to skip this check
  • Add the skip check parameter to the post-build.yaml, and update the branches from the list above
  • Update the publishing pipeline yaml

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. I think for completeness that's probably a better solution.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it might be a good idea to split the SkipSafetyChecks property in two, one for publishing to stable feeds, and one for publishing builds from internal repos to public feeds?
I'd hardcode the publishing one to true, until I go and update all internal repos that publish to public, and then I'd let it be configured by the darc call as a parameter

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep that sounds good to me.

$isInternalBuild = $true
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be a bit safer to start with $true and set to $false? Then we'd have a false positive the other way and worst case we block a public build but not push an internal one to public feeds by mistake (e.g. this condition fails or the build info gets changed..)


# Set tags on publishing for visibility
Write-Host "##vso[build.updatebuildnumber]$buildNumberName"
Write-Host "##vso[build.addbuildtag]Channel(s) - $channelNames"
Expand All @@ -71,6 +76,7 @@ try {
Write-Host "##vso[task.setvariable variable=AzDOBuildId]$($buildInfo.azureDevOpsBuildId)"
Write-Host "##vso[task.setvariable variable=AzDOAccount]$($buildInfo.azureDevOpsAccount)"
Write-Host "##vso[task.setvariable variable=AzDOBranch]$($buildInfo.azureDevOpsBranch)"
Write-Host "##vso[task.setvariable variable=IsInternalBuild]$isInternalBuild"
}
catch {
Write-Host $_
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ public async Task PushNugetPackageTestsAsync(int pushAttemptsBeforeSuccess, bool
// Functionality is the same as this is in the base class, create a v2 object to test.
var task = new PublishArtifactsInManifestV3
{
InternalBuild = true,
IsInternalBuild = true,
BuildEngine = buildEngine,
NugetPath = fakeNugetExeName,
MaxRetryCount = 5, // In case the default changes, lock to 5 so the test data works
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ public class PublishArtifactsInManifest : MSBuildTaskBase
/// Whether this build is internal or not. If true, extra checks are done to avoid accidental
/// publishing of assets to public feeds or storage accounts.
/// </summary>
public bool InternalBuild { get; set; }
public bool IsInternalBuild { get; set; }

public bool PublishInstallersAndChecksums { get; set; } = false;

Expand Down Expand Up @@ -347,7 +347,7 @@ internal PublishArtifactsInManifestBase ConstructPublishingV3Task(BuildModel bui
MaestroApiEndpoint = this.MaestroApiEndpoint,
BuildAssetRegistryToken = this.BuildAssetRegistryToken,
NugetPath = this.NugetPath,
InternalBuild = this.InternalBuild,
IsInternalBuild = this.IsInternalBuild,
SkipSafetyChecks = this.SkipSafetyChecks,
AkaMSClientId = this.AkaMSClientId,
AkaMSClientCertificate = !string.IsNullOrEmpty(AkaMSClientCertificate) ?
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ public abstract class PublishArtifactsInManifestBase : Microsoft.Build.Utilities
/// Whether this build is internal or not. If true, extra checks are done to avoid accidental
/// publishing of assets to public feeds or storage accounts.
/// </summary>
public bool InternalBuild { get; set; } = false;
public bool IsInternalBuild { get; set; } = false;

/// <summary>
/// If true, safety checks only print messages and do not error
Expand Down Expand Up @@ -349,17 +349,20 @@ protected async Task PersistPendingAssetLocationAsync(IMaestroApi client)
}

/// <summary>
/// Protect against accidental publishing of internal assets to non-internal feeds.
/// Run a check to verify that we're not publishing an internal build to non-internal feeds.
/// </summary>
protected void CheckForInternalBuildsOnPublicFeeds(TargetFeedConfig feedConfig)
protected void CheckForInternalBuildsOnPublicFeeds()
{
// If separated out for clarity.
if (!SkipSafetyChecks)
if (!SkipSafetyChecks && IsInternalBuild)
{
if (InternalBuild && !feedConfig.Internal)
foreach (TargetFeedConfig feedConfig in FeedConfigs.Values.SelectMany(x => x))
{
Log.LogError($"Use of non-internal feed '{feedConfig.TargetURL}' is invalid for an internal build. This can be overridden with '{nameof(SkipSafetyChecks)}= true'");
}
if (feedConfig.Internal == false)
{
Log.LogError($"Internal builds shouldn't be published to public feed '{feedConfig.TargetURL}'. This behavior can be overridden with '{nameof(SkipSafetyChecks)}= true'");
}
}
dkurepa marked this conversation as resolved.
Show resolved Hide resolved
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,7 @@ public override async Task<bool> ExecuteAsync()
}

CheckForStableAssetsInNonIsolatedFeeds();
CheckForInternalBuildsOnPublicFeeds();

if (Log.HasLoggedErrors)
{
Expand Down
Loading