Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[main] Update dependencies from dotnet/arcade #251

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
3edc48c
Update dependencies from https://github.com/dotnet/arcade build 20240…
dotnet-maestro[bot] Aug 12, 2024
4d0105d
Update dependencies from https://github.com/dotnet/arcade build 20240…
dotnet-maestro[bot] Aug 19, 2024
a2ab804
Update dependencies from https://github.com/dotnet/arcade build 20240…
dotnet-maestro[bot] Aug 21, 2024
40160a2
Update dependencies from https://github.com/dotnet/arcade build 20240…
dotnet-maestro[bot] Aug 26, 2024
01f3c80
Update dependencies from https://github.com/dotnet/arcade build 20240…
dotnet-maestro[bot] Sep 2, 2024
5b3eafd
Update dependencies from https://github.com/dotnet/arcade build 20240…
dotnet-maestro[bot] Sep 9, 2024
1b7dcd5
Update dependencies from https://github.com/dotnet/arcade build 20240…
dotnet-maestro[bot] Sep 16, 2024
c84b1e5
Update dependencies from https://github.com/dotnet/arcade build 20240…
dotnet-maestro[bot] Sep 23, 2024
270e285
Update dependencies from https://github.com/dotnet/arcade build 20240…
dotnet-maestro[bot] Sep 30, 2024
c68bd5b
Update dependencies from https://github.com/dotnet/arcade build 20241…
dotnet-maestro[bot] Oct 7, 2024
0909b39
Update dependencies from https://github.com/dotnet/arcade build 20241…
dotnet-maestro[bot] Oct 10, 2024
60292b4
Update dependencies from https://github.com/dotnet/arcade build 20241…
dotnet-maestro[bot] Oct 14, 2024
013710d
Update dependencies from https://github.com/dotnet/arcade build 20241…
dotnet-maestro[bot] Oct 21, 2024
1e1e0eb
Update dependencies from https://github.com/dotnet/arcade build 20241…
dotnet-maestro[bot] Oct 28, 2024
9639b10
Update dependencies from https://github.com/dotnet/arcade build 20241…
dotnet-maestro[bot] Nov 4, 2024
26bc20a
Update dependencies from https://github.com/dotnet/arcade build 20241…
dotnet-maestro[bot] Nov 11, 2024
ea703f7
Update dependencies from https://github.com/dotnet/arcade build 20241…
dotnet-maestro[bot] Nov 18, 2024
3604517
Update dependencies from https://github.com/dotnet/arcade build 20241…
dotnet-maestro[bot] Nov 25, 2024
73a2236
Update dependencies from https://github.com/dotnet/arcade build 20241…
dotnet-maestro[bot] Dec 2, 2024
3cf098a
Update dependencies from https://github.com/dotnet/arcade build 20241…
dotnet-maestro[bot] Dec 9, 2024
fc22727
Update dependencies from https://github.com/dotnet/arcade build 20241…
dotnet-maestro[bot] Dec 16, 2024
937a94f
Update dependencies from https://github.com/dotnet/arcade build 20241…
dotnet-maestro[bot] Dec 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions eng/Version.Details.xml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
<ProductDependencies>
</ProductDependencies>
<ToolsetDependencies>
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="9.0.0-beta.24401.1">
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="10.0.0-beta.24622.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>a3dae8d4fd5a17c147cbecfd31e61463731ac0cc</Sha>
<Sha>8589bd2a216025d753b5f107081bfa28a2d51bb8</Sha>
</Dependency>
</ToolsetDependencies>
</Dependencies>
2 changes: 1 addition & 1 deletion eng/common/SetupNugetSources.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ if ($dotnet31Source -ne $null) {
AddPackageSource -Sources $sources -SourceName "dotnet3.1-internal-transport" -SourceEndPoint "https://pkgs.dev.azure.com/dnceng/_packaging/dotnet3.1-internal-transport/nuget/v2" -Creds $creds -Username $userName -pwd $Password
}

$dotnetVersions = @('5','6','7','8')
$dotnetVersions = @('5','6','7','8','9')

foreach ($dotnetVersion in $dotnetVersions) {
$feedPrefix = "dotnet" + $dotnetVersion;
Expand Down
2 changes: 1 addition & 1 deletion eng/common/SetupNugetSources.sh
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ if [ "$?" == "0" ]; then
PackageSources+=('dotnet3.1-internal-transport')
fi

DotNetVersions=('5' '6' '7' '8')
DotNetVersions=('5' '6' '7' '8' '9')

for DotNetVersion in ${DotNetVersions[@]} ; do
FeedPrefix="dotnet${DotNetVersion}";
Expand Down
1 change: 0 additions & 1 deletion eng/common/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,6 @@ function Build {
/p:Restore=$restore \
/p:Build=$build \
/p:DotNetBuildRepo=$product_build \
/p:ArcadeBuildFromSource=$source_build \
/p:DotNetBuildSourceOnly=$source_build \
/p:Rebuild=$rebuild \
/p:Test=$test \
Expand Down
33 changes: 10 additions & 23 deletions eng/common/core-templates/job/job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ parameters:
# publishing defaults
artifacts: ''
enableMicrobuild: false
enableMicrobuildForMacAndLinux: false
enablePublishBuildArtifacts: false
enablePublishBuildAssets: false
enablePublishTestResults: false
Expand All @@ -33,11 +34,6 @@ parameters:
artifactPublishSteps: []
runAsPublic: false

# Sbom related params
enableSbom: true
PackageVersion: 9.0.0
BuildDropPath: '$(Build.SourcesDirectory)/artifacts'

# 1es specific parameters
is1ESPipeline: ''

Expand Down Expand Up @@ -132,18 +128,11 @@ jobs:
- ${{ preStep }}

- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
- task: MicroBuildSigningPlugin@4
displayName: Install MicroBuild plugin
inputs:
signType: $(_SignType)
zipSources: false
feedSource: https://dnceng.pkgs.visualstudio.com/_packaging/MicroBuildToolset/nuget/v3/index.json
env:
TeamName: $(_TeamName)
MicroBuildOutputFolderOverride: '$(Agent.TempDirectory)'
- template: /eng/common/core-templates/steps/install-microbuild.yml
parameters:
enableMicrobuild: ${{ parameters.enableMicrobuild }}
enableMicrobuildForMacAndLinux: ${{ parameters.enableMicrobuildForMacAndLinux }}
continueOnError: ${{ parameters.continueOnError }}
condition: and(succeeded(), in(variables['_SignType'], 'real', 'test'), eq(variables['Agent.Os'], 'Windows_NT'))

- ${{ if and(eq(parameters.runAsPublic, 'false'), eq(variables['System.TeamProject'], 'internal')) }}:
- task: NuGetAuthenticate@1
Expand Down Expand Up @@ -172,14 +161,12 @@ jobs:
- ${{ each step in parameters.componentGovernanceSteps }}:
- ${{ step }}

- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: MicroBuildCleanup@1
displayName: Execute Microbuild cleanup tasks
condition: and(always(), in(variables['_SignType'], 'real', 'test'), eq(variables['Agent.Os'], 'Windows_NT'))
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- template: /eng/common/core-templates/steps/cleanup-microbuild.yml
parameters:
enableMicrobuild: ${{ parameters.enableMicrobuild }}
enableMicrobuildForMacAndLinux: ${{ parameters.enableMicrobuildForMacAndLinux }}
continueOnError: ${{ parameters.continueOnError }}
env:
TeamName: $(_TeamName)

# Publish test results
- ${{ if or(and(eq(parameters.enablePublishTestResults, 'true'), eq(parameters.testResultsFormat, '')), eq(parameters.testResultsFormat, 'xunit')) }}:
Expand Down
5 changes: 3 additions & 2 deletions eng/common/core-templates/job/source-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,10 @@ parameters:
# The name of the job. This is included in the job ID.
# targetRID: ''
# The name of the target RID to use, instead of the one auto-detected by Arcade.
# nonPortable: false
# portableBuild: false
# Enables non-portable mode. This means a more specific RID (e.g. fedora.32-x64 rather than
# linux-x64), and compiling against distro-provided packages rather than portable ones.
# linux-x64), and compiling against distro-provided packages rather than portable ones. The
# default is portable mode.
# skipPublishValidation: false
# Disables publishing validation. By default, a check is performed to ensure no packages are
# published by source-build.
Expand Down
47 changes: 5 additions & 42 deletions eng/common/core-templates/job/source-index-stage1.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@
parameters:
runAsPublic: false
sourceIndexUploadPackageVersion: 2.0.0-20240522.1
sourceIndexProcessBinlogPackageVersion: 1.0.1-20240522.1
sourceIndexPackageSource: https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json
sourceIndexBuildCommand: powershell -NoLogo -NoProfile -ExecutionPolicy Bypass -Command "eng/common/build.ps1 -restore -build -binarylog -ci"
preSteps: []
binlogPath: artifacts/log/Debug/Build.binlog
Expand All @@ -16,12 +13,6 @@ jobs:
dependsOn: ${{ parameters.dependsOn }}
condition: ${{ parameters.condition }}
variables:
- name: SourceIndexUploadPackageVersion
value: ${{ parameters.sourceIndexUploadPackageVersion }}
- name: SourceIndexProcessBinlogPackageVersion
value: ${{ parameters.sourceIndexProcessBinlogPackageVersion }}
- name: SourceIndexPackageSource
value: ${{ parameters.sourceIndexPackageSource }}
- name: BinlogPath
value: ${{ parameters.binlogPath }}
- template: /eng/common/core-templates/variables/pool-providers.yml
Expand All @@ -34,48 +25,20 @@ jobs:
pool:
${{ if eq(variables['System.TeamProject'], 'public') }}:
name: $(DncEngPublicBuildPool)
image: 1es-windows-2022-open
os: windows
image: windows.vs2022.amd64.open
${{ if eq(variables['System.TeamProject'], 'internal') }}:
name: $(DncEngInternalBuildPool)
image: 1es-windows-2022
os: windows
image: windows.vs2022.amd64

steps:
- ${{ if eq(parameters.is1ESPipeline, '') }}:
- 'Illegal entry point, is1ESPipeline is not defined. Repository yaml should not directly reference templates in core-templates folder.': error

- ${{ each preStep in parameters.preSteps }}:
- ${{ preStep }}

- task: UseDotNet@2
displayName: Use .NET 8 SDK
inputs:
packageType: sdk
version: 8.0.x
installationPath: $(Agent.TempDirectory)/dotnet
workingDirectory: $(Agent.TempDirectory)

- script: |
$(Agent.TempDirectory)/dotnet/dotnet tool install BinLogToSln --version $(sourceIndexProcessBinlogPackageVersion) --add-source $(SourceIndexPackageSource) --tool-path $(Agent.TempDirectory)/.source-index/tools
$(Agent.TempDirectory)/dotnet/dotnet tool install UploadIndexStage1 --version $(sourceIndexUploadPackageVersion) --add-source $(SourceIndexPackageSource) --tool-path $(Agent.TempDirectory)/.source-index/tools
displayName: Download Tools
# Set working directory to temp directory so 'dotnet' doesn't try to use global.json and use the repo's sdk.
workingDirectory: $(Agent.TempDirectory)

- script: ${{ parameters.sourceIndexBuildCommand }}
displayName: Build Repository

- script: $(Agent.TempDirectory)/.source-index/tools/BinLogToSln -i $(BinlogPath) -r $(Build.SourcesDirectory) -n $(Build.Repository.Name) -o .source-index/stage1output
displayName: Process Binlog into indexable sln

- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: AzureCLI@2
displayName: Log in to Azure and upload stage1 artifacts to source index
inputs:
azureSubscription: 'SourceDotNet Stage1 Publish'
addSpnToEnvironment: true
scriptType: 'ps'
scriptLocation: 'inlineScript'
inlineScript: |
$(Agent.TempDirectory)/.source-index/tools/UploadIndexStage1 -i .source-index/stage1output -n $(Build.Repository.Name) -s netsourceindexstage1 -b stage1
- template: /eng/common/core-templates/steps/source-index-stage1-publish.yml
parameters:
binLogPath: ${{ parameters.binLogPath }}
28 changes: 28 additions & 0 deletions eng/common/core-templates/steps/cleanup-microbuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
parameters:
# Enable cleanup tasks for MicroBuild
enableMicrobuild: false
# Enable cleanup tasks for MicroBuild on Mac and Linux
# Will be ignored if 'enableMicrobuild' is false or 'Agent.Os' is 'Windows_NT'
enableMicrobuildForMacAndLinux: false
continueOnError: false

steps:
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
- task: MicroBuildCleanup@1
displayName: Execute Microbuild cleanup tasks
condition: and(
always(),
or(
and(
eq(variables['Agent.Os'], 'Windows_NT'),
in(variables['_SignType'], 'real', 'test')
),
and(
${{ eq(parameters.enableMicrobuildForMacAndLinux, true) }},
ne(variables['Agent.Os'], 'Windows_NT'),
eq(variables['_SignType'], 'real')
)
))
continueOnError: ${{ parameters.continueOnError }}
env:
TeamName: $(_TeamName)
73 changes: 73 additions & 0 deletions eng/common/core-templates/steps/install-microbuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
parameters:
# Enable install tasks for MicroBuild
enableMicrobuild: false
# Enable install tasks for MicroBuild on Mac and Linux
# Will be ignored if 'enableMicrobuild' is false or 'Agent.Os' is 'Windows_NT'
enableMicrobuildForMacAndLinux: false
# Location of the MicroBuild output folder
microBuildOutputFolder: '$(Agent.TempDirectory)'
continueOnError: false

steps:
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
- ${{ if eq(parameters.enableMicrobuildForMacAndLinux, 'true') }}:
# Install Python 3.12.x on when Python > 3.12.x is installed - https://github.com/dotnet/source-build/issues/4802
- script: |
version=$(python3 --version | awk '{print $2}')
major=$(echo $version | cut -d. -f1)
minor=$(echo $version | cut -d. -f2)

installPython=false
if [ "$major" -gt 3 ] || { [ "$major" -eq 3 ] && [ "$minor" -gt 12 ]; }; then
installPython=true
fi

echo "Python version: $version."
echo "Install Python 3.12.x: $installPython."
echo "##vso[task.setvariable variable=installPython;isOutput=true]$installPython"
name: InstallPython
displayName: 'Determine Python installation'
condition: and(succeeded(), ne(variables['Agent.Os'], 'Windows_NT'))

- task: UsePythonVersion@0
inputs:
versionSpec: '3.12.x'
displayName: 'Use Python 3.12.x'
condition: and(succeeded(), eq(variables['InstallPython.installPython'], 'true'), ne(variables['Agent.Os'], 'Windows_NT'))

# Needed to download the MicroBuild plugin nupkgs on Mac and Linux when nuget.exe is unavailable
- task: UseDotNet@2
displayName: Install .NET 8.0 SDK for MicroBuild Plugin
inputs:
packageType: sdk
version: 8.0.x
installationPath: ${{ parameters.microBuildOutputFolder }}/dotnet
workingDirectory: ${{ parameters.microBuildOutputFolder }}
condition: and(succeeded(), ne(variables['Agent.Os'], 'Windows_NT'))

- task: MicroBuildSigningPlugin@4
displayName: Install MicroBuild plugin
inputs:
signType: $(_SignType)
zipSources: false
feedSource: https://dnceng.pkgs.visualstudio.com/_packaging/MicroBuildToolset/nuget/v3/index.json
${{ if and(eq(parameters.enableMicrobuildForMacAndLinux, 'true'), ne(variables['Agent.Os'], 'Windows_NT')) }}:
azureSubscription: 'MicroBuild Signing Task (DevDiv)'
env:
TeamName: $(_TeamName)
MicroBuildOutputFolderOverride: ${{ parameters.microBuildOutputFolder }}
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
continueOnError: ${{ parameters.continueOnError }}
condition: and(
succeeded(),
or(
and(
eq(variables['Agent.Os'], 'Windows_NT'),
in(variables['_SignType'], 'real', 'test')
),
and(
${{ eq(parameters.enableMicrobuildForMacAndLinux, true) }},
ne(variables['Agent.Os'], 'Windows_NT'),
eq(variables['_SignType'], 'real')
)
))
3 changes: 3 additions & 0 deletions eng/common/core-templates/steps/publish-logs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,9 @@ steps:
'$(akams-client-id)'
'$(microsoft-symbol-server-pat)'
'$(symweb-symbol-server-pat)'
'$(dnceng-symbol-server-pat)'
'$(dn-bot-all-orgs-build-rw-code-rw)'
'$(System.AccessToken)'
${{parameters.CustomSensitiveDataList}}
continueOnError: true
condition: always()
Expand All @@ -45,6 +47,7 @@ steps:
SourceFolder: '$(Build.SourcesDirectory)/PostBuildLogs'
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)/PostBuildLogs'
condition: always()

- template: /eng/common/core-templates/steps/publish-build-artifacts.yml
parameters:
Expand Down
8 changes: 6 additions & 2 deletions eng/common/core-templates/steps/source-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,11 @@ steps:
assetManifestFileName=SourceBuild_${{ parameters.platform.name }}.xml
fi

portableBuildArgs=
if [ '${{ parameters.platform.portableBuild }}' != '' ]; then
portableBuildArgs='/p:PortableBuild=${{ parameters.platform.portableBuild }}'
fi

${{ coalesce(parameters.platform.buildScript, './build.sh') }} --ci \
--configuration $buildConfig \
--restore --build --pack $publishArgs -bl \
Expand All @@ -85,8 +90,7 @@ steps:
$targetRidArgs \
$runtimeOsArgs \
$baseOsArgs \
/p:SourceBuildNonPortable=${{ parameters.platform.nonPortable }} \
/p:ArcadeBuildFromSource=true \
$portableBuildArgs \
/p:DotNetBuildSourceOnly=true \
/p:DotNetBuildRepo=true \
/p:AssetManifestFileName=$assetManifestFileName
Expand Down
35 changes: 35 additions & 0 deletions eng/common/core-templates/steps/source-index-stage1-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
parameters:
sourceIndexUploadPackageVersion: 2.0.0-20240522.1
sourceIndexProcessBinlogPackageVersion: 1.0.1-20240522.1
sourceIndexPackageSource: https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json
binlogPath: artifacts/log/Debug/Build.binlog

steps:
- task: UseDotNet@2
displayName: "Source Index: Use .NET 8 SDK"
inputs:
packageType: sdk
version: 8.0.x
installationPath: $(Agent.TempDirectory)/dotnet
workingDirectory: $(Agent.TempDirectory)

- script: |
$(Agent.TempDirectory)/dotnet/dotnet tool install BinLogToSln --version ${{parameters.sourceIndexProcessBinlogPackageVersion}} --add-source ${{parameters.SourceIndexPackageSource}} --tool-path $(Agent.TempDirectory)/.source-index/tools
$(Agent.TempDirectory)/dotnet/dotnet tool install UploadIndexStage1 --version ${{parameters.sourceIndexUploadPackageVersion}} --add-source ${{parameters.SourceIndexPackageSource}} --tool-path $(Agent.TempDirectory)/.source-index/tools
displayName: "Source Index: Download netsourceindex Tools"
# Set working directory to temp directory so 'dotnet' doesn't try to use global.json and use the repo's sdk.
workingDirectory: $(Agent.TempDirectory)

- script: $(Agent.TempDirectory)/.source-index/tools/BinLogToSln -i ${{parameters.BinlogPath}} -r $(Build.SourcesDirectory) -n $(Build.Repository.Name) -o .source-index/stage1output
displayName: "Source Index: Process Binlog into indexable sln"

- ${{ if and(ne(parameters.runAsPublic, 'true'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- task: AzureCLI@2
displayName: "Source Index: Upload Source Index stage1 artifacts to Azure"
inputs:
azureSubscription: 'SourceDotNet Stage1 Publish'
addSpnToEnvironment: true
scriptType: 'ps'
scriptLocation: 'inlineScript'
inlineScript: |
$(Agent.TempDirectory)/.source-index/tools/UploadIndexStage1 -i .source-index/stage1output -n $(Build.Repository.Name) -s netsourceindexstage1 -b stage1
Loading
Loading