arduino / arduino-cli Public

* Simplified implementation of Tool.LatestRelease/GetLatestInstalled methods

* Some code make-up

…orkflow (#1442)

A bug introduced in the 3.9.7 release of Python causes a spurious failure of the
`test\test_lib.py::test_install_git_invalid_library` integration test:
https://bugs.python.org/issue45121

As a workaround, the Python version used by the "Test Integration" CI workflow that runs the integration tests is pinned
to the last working version: 3.9.6. Since it is convenient to get automatic updates for Python patch releases, this full
pin should be reverted back to the "3.9" minor version pin once a new version of Python is released with the bug fixed
and added to versions available for installation via the `actions/setup-python` GitHub Actions action.

… Fixed i18n early static initialization problem (#1425)

* Removed unnecessary fmt.*printf

* removed unused variable

* Fixed i18n early static initialization problem

#1425 (comment)

these particular strings are not correctly handled by the i18n package because
they are declared at package level before the call to i18n.Init(), and so are
just returned as-is.

* Fix lint errors and use error.Is instead of direct comparison

* Revert "guard" in Tr function

otherwise the unit-tests will fail:

=== RUN   TestBoardOptions
--- FAIL: TestBoardOptions (0.00s)
panic: i18n.Tr called before i18n.Init() [recovered]
	panic: i18n.Tr called before i18n.Init()

goroutine 9 [running]:
testing.tRunner.func1.2(0xc56e00, 0xeaee20)
	/opt/hostedtoolcache/go/1.16.7/x64/src/testing/testing.go:1143 +0x332
testing.tRunner.func1(0xc0002a2f00)
	/opt/hostedtoolcache/go/1.16.7/x64/src/testing/testing.go:1146 +0x4b6
panic(0xc56e00, 0xeaee20)
	/opt/hostedtoolcache/go/1.16.7/x64/src/runtime/panic.go:965 +0x1b9
github.com/arduino/arduino-cli/i18n.Tr(0xdceacd, 0x28, 0x0, 0x0, 0x0, 0x0, 0x1)
	/home/runner/work/arduino-cli/arduino-cli/i18n/i18n.go:54 +0xd9
github.com/arduino/arduino-cli/arduino/cores.(*Board).GetBuildProperties(0x13bf060, 0xc0003b6150, 0xc000339400, 0x0, 0x0)
	/home/runner/work/arduino-cli/arduino-cli/arduino/cores/board.go:109 +0x69d
github.com/arduino/arduino-cli/arduino/cores.(*Board).GeneratePropertiesForConfiguration(0x13bf060, 0xdb5f54, 0xe, 0xc000309ef0, 0xc911e0, 0xc0003b6000)
	/home/runner/work/arduino-cli/arduino-cli/arduino/cores/board.go:141 +0x28f
github.com/arduino/arduino-cli/arduino/cores.TestBoardOptions(0xc0002a2f00)
	/home/runner/work/arduino-cli/arduino-cli/arduino/cores/board_test.go:298 +0x4bd
testing.tRunner(0xc0002a2f00, 0xe052a8)
	/opt/hostedtoolcache/go/1.16.7/x64/src/testing/testing.go:1193 +0xef
created by testing.(*T).Run
	/opt/hostedtoolcache/go/1.16.7/x64/src/testing/testing.go:1238 +0x2b3
FAIL	github.com/arduino/arduino-cli/arduino/cores	0.021s
=== RUN   TestIndexParsing
--- FAIL: TestIndexParsing (0.00s)
panic: i18n.Tr called before i18n.Init() [recovered]
	panic: i18n.Tr called before i18n.Init()

* Apply suggestions from code review

Co-authored-by: per1234 <[email protected]>

Co-authored-by: per1234 <[email protected]>

* [skip changelog] Fix Windows integration tests

* Update test/test_sketch.py

Co-authored-by: per1234 <[email protected]>

Co-authored-by: per1234 <[email protected]>

* bump github.com/spf13/cobra to 1.2.1

* go mod tidy

* fix CI failing

* update powershell tests, in new version of cobra the completion file is different

#1429)

* Fix upload command not working as expected when certain flags are used

* Fix absurd upload corner case

…led (#1428)

* Fix integration tests detected_boards fixture

* Fix builtin discoveries not loaded if no platform is installed

* [skip changelog] Add missing breaking change to UPGRADING.md

* [skip changelog] Fix board list test running on CI

* [skip changelog] Fix UPGRADING.md

* [skip changelog] Skip boards without FQBN in tests

* [skip changelog] Fix esp32 integration test on darwin

* Add ways to let users verify if new CLI released

* Code review fixes

Co-authored-by: per1234 <[email protected]>

* Enhance docs

Co-authored-by: per1234 <[email protected]>

* Fix version check for git-snapshots and nightlies

* Change method to request latest release

* Remove ansi library in favor of color

* Fix go mod errors

* Remove useless function

Co-authored-by: Cristian Maglie <[email protected]>

Co-authored-by: per1234 <[email protected]>
Co-authored-by: Cristian Maglie <[email protected]>

* Proper gRPC error handling

* Update gRPC API of board command to return status.Status instead of error

* Update gRPC API of remaining commands to return status.Status instead of error

* Replace custom error with protobuf message

Previously, a custom error was returned when attempting to upgrade a platform that was already at the latest available
version. There is dedicated code for handling this specific error.

Now that the function has been changed to return a status.Status instead of error, the previous check for the return
being this error is no longer possible. The capability is restored by replacing the error with a protocol buffer message.

* Handle details of any type in `core.PlatformUpgrade()` status

The status details of the function are used to identify the specific cause of a non-nil status. This is done via a type
assertion. Previously, this type assertion was configured such that a details of any type other than the expected would
result in a panic. At the moment, that will not occur because we only add details of one type. However, the whole point
of the type assertion is to support details of multiple types, and if other types are added a panic will not be the
appropriate behavior.

A better approach is to check the result of the type assertion, handling the non-nil status as a generic error if its
details are of a different type.

* Return nil on program action if an error occurred

* Refactoring 'upload' commands

* Refactoring 'board' commands

* Refactoring 'compile' commands

* Refactoring 'core' commands

* Refactoring 'debug' commands

* Refactoring 'lib' commands

* Refactoring 'sketch' commands

* Refactoring 'commands' commands

* updated tests and fixed some error wording

* fixed go lint warnings

* Apply suggestions from code review

Co-authored-by: per1234 <[email protected]>

* Apply changes from code review

Co-authored-by: Silvano Cerza <[email protected]>

* fix i18n

Co-authored-by: per1234 <[email protected]>
Co-authored-by: Silvano Cerza <[email protected]>

…and `test-legacy` tasks (#1417)

* Bump version of golang in go.mod to 1.16 to align with ditribution containers

* Refactor and add TEST_LDFLAGS to test-unit-race and test-legacy tasks in Taskfile

* Run

* Simplified utils.Match function

* match func doesn't need err anymore

* Removed leftover

* Removed unused mutex

* Factored out all duplicated match helpers

* fix i18n

* Add completion for PowerShell

* Manage command description for PS completion

* Add test for PS completion

* Add PS documentation

* Fix formatting

* Fix python lint for PS tests

* Update docs/command-line-completion.md

Co-authored-by: Silvano Cerza <[email protected]>

Co-authored-by: Silvano Cerza <[email protected]>

… version (#1410)

* [skip changelog] Add DiscoveryManager to PackageManager

* Add loading of PluggableDiscoveries when loading a platform release

* Added compatibility layer for non-pluggable platforms

* Implemented board list with discoveries

* Implemented discovery loading after initialization

* Implemented board watch with discoveries

* Fix load discoveries tests

* Fix some issues with board list watcher

* Fix FindToolsRequiredFromPlatformRelease not returning discoveries

* Enhanced handling of some discoveries states

* Fix PackageManager reset

* Add function to convert discovery.Port to rpc.Port

* Moved reference argument parsing to new package

* Fix functions docstrings

* Remove duplicated code to initialize Sketch path

* Add property conversion for platform not supporting pluggable discovery

* Fix board list watch not working

* Fix crash when converting Port to rpc struct

* Add generic Port argument

* Change gRPC upload functions to use new Port message

* Add support for upload user fields

* Fix upload unit tests

* Fix code naming issues

* Added builtin:mdns-discovery

* Do not panic if discovery tool is not installed

* Implemented port/protocol detection at CLI startup time

* Perform 1200bps-touch only on serial ports

* Added missing properties for pluggable upload

* Correctly implemented 'board list' timeout option

* Updated mdns-discovery to 0.9.2

* Add documentation

* Add board properties to board list command and gRPC function

* Fix documentation and code comments

Co-authored-by: per1234 <[email protected]>

* Fix crash when attempting upload without specifying port address

* Fix unit tests

* Update go-properties-orderedmap to fix discovery properties issues

* Fix more documentation

Co-authored-by: per1234 <[email protected]>

* Clarify pluggable discovery specification

* More documentation fixes

* Add upload_port properties docs in platform specification

* Change links from pluggable discovery RFC to official docs

* Add more upload mock integration tests

* Fix integration tests

* Change property to declare pluggable discoveries

* Change property to declare pluggable discoveries

* Fix documentation

Co-authored-by: per1234 <[email protected]>

* Fix loading of platform not supporting pluggable discovery

* Fix more documentation

Co-authored-by: per1234 <[email protected]>

* Add pluggable discovery states documentation

* Enhanced handling of pluggable discoveries states

* Discoveries processes are now killed if the HELLO command fails

* Add pluggable discovery logging

* Enhanced handling of failing pluggable discoveries

* Fix pluggable discoveries parallelization

* Discoveries event channels are now created when start sync is called

* Cached ports are now reset on discovery stop

* Renamed ListSync methods to ListCachedPorts

* Pluggable discovery upload user fields are now limited to 50 chars

* Fix i18n strings

* Fix failing integration tests

* Fix i18n data

* Fix integration tests again

* [skip changelog] Internationalize strings added for pluggable discovery support (#1384)

* Update docs/pluggable-discovery-specification.md

Co-authored-by: per1234 <[email protected]>

* Fix failing workflows

* Updated upload-mock tests for generation

* Added a lot of mock upload test (also with programmer option)

* test_upload_mock: Handle '{' and '}' in recipes

* network ota: autoconvert network_patter from legacy

* Automatically add port detection properties for network discovery

* Slightly improved 'board list' text output

* Default 'board list' timeout to 1s

* Added some code review fixes

* Added unit test for legacy-package conversion to pluggable discovery

Co-authored-by: Cristian Maglie <[email protected]>
Co-authored-by: per1234 <[email protected]>

…tion (#1404)

Use of the major version ref will cause the workflow to benefit from ongoing development to its `actions/github-script`
GitHub Actions action dependency at each patch or minor release up until such time as a new major release is made.
Dependabot will submit a PR at that time, which will serve as the notification that a newer version is available. At that
time, the maintainer should evaluate whether any changes to the workflow are required by the breaking change that
triggered the major release before manually updating the major ref (i.e., `uses: actions/github-script@v5`).

…1400)

* [skip changelog] Run relevant workflows on release branch creation

The trunk-based development strategy is employed by this repository. This means that the release branch may contain a
subset of the history of the default branch.

The status of the GitHub Actions workflows should be evaluated before making a release. However, this is not so simple as
checking the status of the commit at the tip of the release branch. The reason is that, for the sake of efficiency, the
workflows are configured to run only when the processes are relevant to the trigger event (e.g., no need to run the Go
unit tests for a change to the readme).

In the case of the default branch, you can simply set the workflow runs filter to that branch and then check the result
of the latest run of each workflow of interest. However, that was not possible to do with the release branch since it
might be that the workflow was never run in that branch. The status of the latest run of the workflow in the default
branch might not match the status for the release branch if the release branch does not contain the full history.

For this reason, it will be helpful to trigger all relevant workflows on the creation of a release branch. This will
ensure that each of those workflows will always have at least one run in the release branch. Subsequent commits pushed to
the branch can run based on their usual trigger filters and the status of the latest run of each workflow in the branch
will provide an accurate indication of the state of that branch.

Branches are created for purposes other than releases, most notably feature branches to stage work for a pull request.
Because the workflows are very comprehensive, it would not be convenient or efficient to run them on the creation of
every feature branch.

Unfortunately, GitHub Actions does not support filters on the `create` event of branch creation like it does for the
`push` and `pull_request` events. There is support for a `branches` filter of the `push` event, but that filter is an AND
to the `paths` filter and this application requires an OR. For this reason, the workflows must be triggered by the
creation of any branch. The unwanted job runs are prevented by adding a `run-determination` job with the branch filter
handled by Bash commands. The other jobs of the workflow use this `run-determination` job as a dependency, only running
when it indicates they should via a job output. Because this minimal `run-determination` job runs very quickly, it is
roughly equivalent to the workflow having been skipped entirely for non-release branch creations. This approach has been
in use for some time already in the website deployment workflow.

* [skip changlog] Simplify jobs run determination logic

Co-authored-by: Cristian Maglie <[email protected]>

Co-authored-by: Cristian Maglie <[email protected]>

… fork (#1397)

* [skip changelog] Use major version refs of action in "Test Go" workflow

Use of the major version ref will cause the workflow to benefit from ongoing development to the actions up until such
time as a new major release of an action is made, at which time we would need to evaluate whether any changes to the
workflow are required by the breaking change that triggered the major release before updating the major ref
(e.g., `uses: codecov/codecov-action@v3`).

The previous pin to the patch version required an update to the workflow on every action release in order to keep it
updated.

* [skip changelog] Don't require Codecov upload success for test run in fork

The "Test Go" workflow uploads code coverage data to Codecov. There will occasionally be spurious upload failures caused
by transient network outages. These will typically succeed after the workflow is re-run, but the option to re-run is not
offered when the workflow run passes.

Because it's important that the data be complete, the `codecov/codecov-action` action is configured to fail the workflow
run if the upload does not succeed. However, the upload will never be able to succeed for workflow runs in a fork where
the owner has not set up Codecov. For this reason, the `fail_ci_if_error` input setting is made conditional upon the
repository name.

The result is:

- Coverage data upload success is required for all workflow runs in the `arduino/arduino-cli` repository.
- Uploads are attempted for workflow runs in forks (because the fork owner might have Codecov set up and want the data),
  but they are not required to succeed and will fail silently.

* [skip changelog] Bring shell scripts into compliance with standard formatting

A standardized style for all Arduino Tooling shell scripts has been established. The `.editorconfig` file is updated
accordingly and all the repository's scripts made compliant with it.

* [skip changelog] Sync install script with template

We have assembled a collection of reusable project assets:
https://github.com/arduino/tooling-project-assets
These assets will be used in the repositories of all Arduino tooling projects.

Some minor improvements and standardizations have been made in the upstream "template" installation script, and those are
introduced to this repository via this pull request.

Notable:

- ShellCheck compliance
- Support for specifying nightly build versions (e.g., "nightly-latest") via the script argument
- Remove cryptic output not of interest to the user (likely forgotten debug artifacts from script development)

We have assembled a collection of reusable GitHub Actions workflows:
https://github.com/arduino/tooling-project-assets
These workflows will be used in the repositories of all Arduino tooling projects.

Some minor improvements and standardizations have been made in the upstream "template" workflow, and those are introduced to this repository via this pull request.

Notable:

- Replace changelog file read, deprecated `actions/create-release`, and asset upload steps with the comprehensive `ncipollo/release-action` action