Hmm, this looks like an infra issue given the 404, rather than M1 / Apple Silicon specific.

dotnet-install: Attempting to download using primary link https://dotnetcli.azureedge.net/dotnet/Runtime/2.1.30/dotnet-runtime-2.1.30-osx-arm64.tar.gz
curl: (22) The requested URL returned error: 404

Can you please try again in a few hours?

@TanayParikh I tried running the restore.sh again, and the result was the same.
The problem is that the script is trying to download older runtimes (version 2.1 and 3.1) for Apple M1, and they are not available. When I replace arm64 with x64 in the primary download URL https://dotnetcli.azureedge.net/dotnet/Runtime/2.1.30/dotnet-runtime-2.1.30-osx-arm64.tar.gz I get the tar.gz just fine.
Spending time retrofitting netcore 2.1 and netcore 3.1 for M1 is not a good idea, so we need to change the scripts to pull legacy runtimes for x64. They would run on M1 with Rosetta, which is fine for building and testing.

M1 support should've been covered via #26821 I believe. Thoughts @dougbu?

/cc @dotnet/aspnet-build

Could we add a block to global.json like

That won't work @wtgodbe because the problem is w/ older runtimes that are needed in some of our projects. We're a bit stuck here because we have projects targeting both netcoreapp2.1 and we use the netcoreapp3.1 copy of dotnet-ef.dll in project template tests.

The tooling / task projects could perhaps be disabled when on macOS Arm64 and we might be able to use a different copy of dotnet-ef.dll these days. The problem w/ the first part is the build would be incomplete. I'm not sure the second part is viable.

The other option might be to use more recent TFMs but only when building on macOS Arm64. Would just require a few more properties set w/ appropriate Conditions in eng/Versions.props and used in global.json.

@dougbu Would you suggest a workaround for this? Alternatively, do you think it would be possible to make a fix that would let me run a build just for the Mvc solution with no dependencies? My ultimate goal is to try submitting a pull request for a small enhancement, but I need to be able to build from the sources before I get started.

Not sure but removing the problematic item or items from global.json and running restore.sh and src/Mvc/build.sh with /p:SkipTestBuild=true may work.

@Chrisboh @dotnet/dnceng is there general guidance for building on newer RIDs that we don't have support for on older TFMs? This repo currently doesn't build on Apple Silicon because we specify 2.1 & 3.1 runtimes in global.json

I am not sure if there is any general guidance here. @mmitche do you know?

I don't think there is anything specifically written down, but typically we've had to bootstrap these situations with a preview SDK. Essentially a stage0 type update.

Does that mean adding the rid to Microsoft.Netcore.Platforms and getting that ingested into the SDK? And backporting that to 3.1/5.0/6.0? I guess there isn't a solution for 2.1, maybe we should move our netcoreapp2.1 projects to netstandard

maybe we should move our netcoreapp2.1 projects to netstandard

Is that viable given the OpenAPI "glue" (the main place we use netcoreapp2.1) is tooling i.e. executables that are supposed to work a fair while after we stop supporting the runtime❔ Our OpenAPI code generation partners still supported .NET 1.0 when this was written and netcoreapp2.1 was originally a compromise.

I can think of a few ways forward which leave the OpenAPI stuff mostly alone and avoid creating a preview SDK for a relatively limited case. I may be overly worried about the complexity of that option however.

1. Remove the tools/runtimes/dotnet/2.1.30 and tools/runtimes/aspnetcore/3.1.21 entries from global.json entirely. This would mean we build and test against whatever versions are bundled in the .NET SDK. That might be an issue the longer we go supporting ASP.NET Core 2.1 on full framework (which will last forever). May also run into problems if the currently-bundled runtimes are removed at some point in the future or if it's currently impossible to reference netcoreapp2.1 at all on M1.
2. Conditionalize the versions we use for those hard-coded runtimes and special case platforms (like M1) where they may not be available, e.g. set $(AdditionalDotnetRuntime) to 2.1.30 most of the time but match $(MicrosoftNETCoreBrowserDebugHostTransportVersion) on M1 machines. This amounts to a slight expansion of our current $(MicrosoftNETCoreBrowserDebugHostTransportVersion) property use and, like that, leverages existing Arcade features. One downside would be the "special-casedness" of the option, especially if we believe we're going to hit more problems like this one in the future.

• Another downside is we'd also have to special case the TFMs used in a few of our projects to use runtimes we're sure are available.

3. Don't built the relevant projects on M1 machines and somehow conditionalize the entries in the global.json file. I am not a fan of this because it creates a category of machines that are only partially supported when building the repo. It likely would also require new code in Arcade for the "conditionalize" part.
4. Move tools/runtimes/dotnet/2.1.30 and tools/runtimes/aspnetcore/3.1.21 to tools/runtimes/dotnet/{something}/2.1.30 and tools/runtimes/aspnetcore/{something}/3.1.21. {something} would indicate to Arcade that the platform-specific runtime may not be available and avoid failing if not. Similar to (1), this wouldn't work on its own if the runtime download problem is a symptom of a wider inability to use a TFM.

I would appreciate everyone's thoughts on the various options. Better ideas much appreciated❕

/cc @rafikiassumani-msft because anything we do here may impact Microsoft.Extensions.ApiDescription.Server.

I'm a fan of option 1, if it works - the downlevel versions in the SDK are already very up to date: https://github.com/dotnet/installer/blob/b6c29ddce6f3d0472e3449bb65fef6c08b9bd644/src/redist/targets/GenerateBundledVersions.targets#L27-L32

@dasblinkenlight could you try restoring/building this branch to see if it works on Apple silicon? https://github.com/dotnet/aspnetcore/tree/wtgodbe/Silicon

@wtgodbe I can't get your branch - I see it when I browse to the URL, but trying to fetch gives me a "Permission denied" error:

(aspnetcore) % git remote add wtgodbe [email protected]:wtgodbe/Silicon            
(aspnetcore) % git fetch wtgodbe                                    
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Try (from your aspnetcore directory):

git remote add upstream https://github.com/dotnet/aspnetcore
git fetch upstream
git checkout wtgodbe/Silicon

@wtgodbe This worked perfectly. I started ./build.sh again, and this time it ran to completion. Thank you very much for your help!

That's great!

@dougbu any objections to taking 6a22c03? I'm ok with relying on the implicit versions in the SDK, those stay up-to-date with what we have in global.json anyways.

#40393

@dougbu any objections

None. We're not using much test coverage on the 3.1 side and our 2.1 usage is teensy.

Should work out-of-the-box now as of #40393

I tried to run ./build.sh on a fresh M1 and it failed.

Codes/aspnetcore [main●] » ./eng/build.sh
warning: No default group of projects was specified, so building the 'managed' and its dependent subset of projects. Run build.sh --help for more details.
warning: Building of C# project is enabled and has dependencies on NodeJS projects. Building of NodeJS projects is enabled since node is detected on PATH.
  Determining projects to restore...
  Tool 'dotnet-serve' (version '1.8.15') was restored. Available commands: dotnet-serve
  Tool 'microsoft.playwright.cli' (version '1.2.2') was restored. Available commands: playwright

  Restore was successful.
  All projects are up-to-date for restore.
  Determining projects to restore...
  All projects are up-to-date for restore.
  GenerateFiles -> /Users/soheilalizadeh/Codes/aspnetcore/artifacts/bin/GenerateFiles/Directory.Build.props
  GenerateFiles -> /Users/soheilalizadeh/Codes/aspnetcore/artifacts/bin/GenerateFiles/Directory.Build.targets
Unknown CPU arm64 detected, treating it as x64
Attempting to install dotnet from public_location.
dotnet-install: Note that the intended use of this script is for Continuous Integration (CI) scenarios, where:
dotnet-install: - The SDK needs to be installed without user interaction and without admin rights.
dotnet-install: - The SDK installation doesn't need to persist across multiple CI runs.
dotnet-install: To set up a development environment or to run apps, use installers rather than this script. Visit https://dotnet.microsoft.com/download to get the installer.

...

Build Erros:

/Users/soheilalizadeh/.nuget/packages/grpc.tools/2.40.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003: The specified task executable "/Users/soheilalizadeh/.nuget/packages/grpc.tools/2.40.0/tools/macosx_x64/protoc" could not be run. System.ComponentModel.Win32Exception (86): An error occurred trying to start process '/Users/soheilalizadeh/.nuget/packages/grpc.tools/2.40.0/tools/macosx_x64/protoc' with working directory '/Users/soheilalizadeh/Codes/aspnetcore/src/Grpc/test/testassets/InteropClient'. Bad CPU type in executable [/Users/soheilalizadeh/Codes/aspnetcore/src/Grpc/test/testassets/InteropClient/InteropClient.csproj]
/Users/soheilalizadeh/.nuget/packages/grpc.tools/2.40.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) [/Users/soheilalizadeh/Codes/aspnetcore/src/Grpc/test/testassets/InteropClient/InteropClient.csproj]
/Users/soheilalizadeh/.nuget/packages/grpc.tools/2.40.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) [/Users/soheilalizadeh/Codes/aspnetcore/src/Grpc/test/testassets/InteropClient/InteropClient.csproj]
/Users/soheilalizadeh/.nuget/packages/grpc.tools/2.40.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/Users/soheilalizadeh/Codes/aspnetcore/src/Grpc/test/testassets/InteropClient/InteropClient.csproj]
/Users/soheilalizadeh/.nuget/packages/grpc.tools/2.40.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.Execute() [/Users/soheilalizadeh/Codes/aspnetcore/src/Grpc/test/testassets/InteropClient/InteropClient.csproj]
/Users/soheilalizadeh/.nuget/packages/grpc.tools/2.40.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003: The specified task executable "/Users/soheilalizadeh/.nuget/packages/grpc.tools/2.40.0/tools/macosx_x64/protoc" could not be run. System.ComponentModel.Win32Exception (86): An error occurred trying to start process '/Users/soheilalizadeh/.nuget/packages/grpc.tools/2.40.0/tools/macosx_x64/protoc' with working directory '/Users/soheilalizadeh/Codes/aspnetcore/src/Grpc/test/testassets/InteropWebsite'. Bad CPU type in executable [/Users/soheilalizadeh/Codes/aspnetcore/src/Grpc/test/testassets/InteropWebsite/InteropWebsite.csproj]
/Users/soheilalizadeh/.nuget/packages/grpc.tools/2.40.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) [/Users/soheilalizadeh/Codes/aspnetcore/src/Grpc/test/testassets/InteropWebsite/InteropWebsite.csproj]
/Users/soheilalizadeh/.nuget/packages/grpc.tools/2.40.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) [/Users/soheilalizadeh/Codes/aspnetcore/src/Grpc/test/testassets/InteropWebsite/InteropWebsite.csproj]
/Users/soheilalizadeh/.nuget/packages/grpc.tools/2.40.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/Users/soheilalizadeh/Codes/aspnetcore/src/Grpc/test/testassets/InteropWebsite/InteropWebsite.csproj]
/Users/soheilalizadeh/.nuget/packages/grpc.tools/2.40.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.Execute() [/Users/soheilalizadeh/Codes/aspnetcore/src/Grpc/test/testassets/InteropWebsite/InteropWebsite.csproj]

System Info

Chip: M1 Max

OS Version: macOS 12.2.1 

Regarding the message in the build which says it detected an unknown CPU, I tried to check the source and on my machine uname -m returns arm64 but in the following script, I don't see arm64 in the switch case.

I tried also to change the cpuname, and build errors still resists.

@omajid is the above discrepancy in dotnet-install known?

I wasn't aware of it until now. That part of dotnet-install.sh was adapted from the similar piece of code in dotnet/runtime, in 2019. Later, in 2020, runtime took this change that we should also include in dotnet-install.sh: https://github.com/dotnet/runtime/pull/39142/files#diff-692d7a1cb247ffea0cb746bd6f4c3d576406e279af19a1810c0278ca5f16f28c

Would you like me to do a PR? I dont have macOS arm64 machine to test, though.

I think we should update the global dotnet-install scripts to account for arm64 - I unfortunately also don't have a macOS arm64 test machine. Maybe @dotnet/dnceng could help you access one if need be

@wtgodbe I can do it also if you want, adding arm64| should be enough right?

@xsoheilalizadeh Yes, that would be great!

PR #40698

Thank you for the PR, but that file is centrally managed & shared by many dotnet repos - @omajid will handle making the centralized fix

Had to revert #40393, if we can find a way to use BinLogToSln without always explicitly restoring the 3.1 runtime we should un-revert it

I think we should update the global dotnet-install scripts to account for arm64 - I unfortunately also don't have a macOS arm64 test machine. Maybe @dotnet/dnceng could help you access one if need be

We have and can share an M1 machine on a limited basis.  Just ping @ilyas1974 when you're ready; they're limited so we'd like to minimize downtime if possible.