Skip to content

Commit

Permalink
Forge integration tests (#370)
Browse files Browse the repository at this point in the history
* Add integration test debugging to launch cfg, refactor magefile, and update comments

- Introduced a new launch configuration for "mage RunIntegrationTests" in the VSCode settings.
- Enhanced the documentation and comments within `magefiles/magefile.go` for clarity.

* Expose runIntegrationTests as a Mage Command and Enhance Documentation

- Made `runIntegrationTests` an exported function as `RunIntegrationTests` to allow it to be used as a Mage command.
- Added comprehensive comments to functions related to TTP example README parsing and command execution.

* Enhance compilation and testing functionalities in magefile

- Introduce `Compile` function using `goreleaser` for project compilation.
- Implement environment variable handling for target OS and architecture in compilation.
- Improve the `InstallDeps` function's description and dependencies.
- Handle `pre-commit` tool validation and error messaging in `RunPreCommit`.
- Add environment variable handling and adjustments in `RunIntegrationTests`.
- Implement temporary directory handling for binary during integration tests.

* Refactor magefile and introduce magefiles README

- Introduce a comprehensive README for the `magefiles` to document its functionality and usage.
- Refine the `Compile` function to simplify environment variable handling for OS and architecture.
- Remove unnecessary environment variables in `RunIntegrationTests` and ensure a cleaner flow for binary path setting.
- Introduce utility functions `changeToRepoRoot` and `getBinaryDirName` for cleaner and modular code.
- Add appropriate cleanup for generated directories and files post integration tests.

* Optimize magefile structure and introduce testing script

- Organize build tags for the `magefile`.
- Remove redundant and unused `FindExportedFuncsWithoutTests` function.
- Streamline the `RunTests` function by removing unnecessary dependencies.
- Introduce `magefile_test.sh` script to automate the testing of magefile commands, ensuring each command works as expected.

* Update developer documentation with mage test command; linting

- Add a section in `docs/dev/README.md` to guide developers on testing all mage functions using the new script.
- Linted docs/dev/update-go.md

* Refactor Compile function for better usability and clarity

- Removed direct parameter from the Compile function and shifted to an environment variable approach for the 'release' variable.
- Updated function comments to be more concise and informative, adhering to the 80-character line limit.
- Adjusted the RunIntegrationTests function to set the 'release' environment variable before calling Compile.

* Refactor magefiles README doc gen; pin afero dep

- Simplify the introduction in the README for magefiles.
- Reorganize the magefile function comments to provide clearer examples and more consistent formatting.
- Update the docgen README template to support magefiles.
- Pin afero to desired version

* Update pre-commit hooks

* Update magefile_test with final changes to compile command
  • Loading branch information
l50 authored Oct 10, 2023
1 parent 7efe2bc commit be4b0a5
Show file tree
Hide file tree
Showing 11 changed files with 737 additions and 481 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ coverage-all.out
# Compiled binary
ttpforge
ttpforge-windows
dist

# vscode debugger
__debug_bin
Expand Down
11 changes: 11 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,17 @@
"preLaunchTask": "create mage_output_file.go and run code to debug",
"postDebugTask": "delete mage_output_file.go"
},
{
"name": "Mage RunIntegrationTests",
"type": "go",
"request": "launch",
"mode": "auto",
"program": "${workspaceFolder}/magefiles/mage_output_file.go",
"args": ["runIntegrationTests"],
"buildFlags": "magefile.go",
"preLaunchTask": "create mage_output_file.go and run code to debug",
"postDebugTask": "delete mage_output_file.go"
},
{
"type": "go",
"request": "launch",
Expand Down
9 changes: 9 additions & 0 deletions docs/dev/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,3 +120,12 @@ Once you want to get feedback from pre-commit, reinstall it with:
```bash
pre-commit install
```

### Test all mage functions

Run the following command to execute all mage functions
and ensure they are working as expected:

```bash
bash magefiles/scripts/magefile_test.sh
```
11 changes: 6 additions & 5 deletions docs/dev/update-go.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@
If you want to change the supported/recommended Golang version(s) for TTPForge,
you must do all of the following:

* Update minimum required Golang version specified in `go.mod`
* Make a corresponding update to `magefiles/go.mod`
* Update the version referenced in `.devcontainer/bash/Dockerfile` (example: `FROM golang:1.21.1-alpine`)
* Update recommended Golang version for asdf in `.tool-versions`
* Update the [developer documentation](README.md) to indicate the new version(s)
- Update minimum required Golang version specified in `go.mod`
- Make a corresponding update to `magefiles/go.mod`
- Update the version referenced in `.devcontainer/bash/Dockerfile` (example:
`FROM golang:1.21.1-alpine`)
- Update recommended Golang version for asdf in `.tool-versions`
- Update the [developer documentation](README.md) to indicate the new version(s)
2 changes: 2 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,5 @@ require (
golang.org/x/text v0.9.0 // indirect
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
)

replace github.com/spf13/afero => github.com/spf13/afero v1.2.1
181 changes: 181 additions & 0 deletions magefiles/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
# TTPForge/magefiles

`magefiles` provides utilities that would normally be managed
and executed with a `Makefile`. Instead of being written in the make language,
magefiles are crafted in Go and leverage the [Mage](https://magefile.org/) library.

---

## Table of contents

- [Functions](#functions)
- [Contributing](#contributing)
- [License](#license)

---

## Functions

### Compile()

```go
Compile() error
```

Compile compiles the Go project using goreleaser. The behavior is
controlled by the 'release' environment variable. If the GOOS and
GOARCH environment variables are not set, the function defaults
to the current system's OS and architecture.

**Environment Variables:**

release: Determines the compilation mode.

If "true", compiles all supported releases for TTPForge.
If "false", compiles only the binary for the specified OS
and architecture (based on GOOS and GOARCH) or the current
system's default if the vars aren't set.

GOOS: Target operating system for compilation. Defaults to the
current system's OS if not set.

GOARCH: Target architecture for compilation. Defaults to the
current system's architecture if not set.

Example usage:

```go
release=true mage compile # Compiles all supported releases for TTPForge
GOOS=darwin GOARCH=arm64 mage compile false # Compiles the binary for darwin/arm64
GOOS=linux GOARCH=amd64 mage compile false # Compiles the binary for linux/amd64
```

**Returns:**

error: An error if any issue occurs during compilation.

---

### GeneratePackageDocs()

```go
GeneratePackageDocs() error
```

GeneratePackageDocs creates documentation for the various packages
in the project.

Example usage:

```go
mage generatepackagedocs
```

**Returns:**

error: An error if any issue occurs during documentation generation.

---

### InstallDeps()

```go
InstallDeps() error
```

InstallDeps installs the Go dependencies necessary for developing
on the project.

Example usage:

```go
mage installdeps
```

**Returns:**

error: An error if any issue occurs while trying to
install the dependencies.

---

### RunIntegrationTests()

```go
RunIntegrationTests() error
```

RunIntegrationTests executes all integration tests by extracting the commands
described in README files of TTP examples and then executing them.

Example usage:

```go
mage runintegrationtests
```

**Returns:**

error: An error if any issue occurs while running the tests.

---

### RunPreCommit()

```go
RunPreCommit() error
```

RunPreCommit updates, clears, and executes all pre-commit hooks
locally. The function follows a three-step process:

First, it updates the pre-commit hooks.
Next, it clears the pre-commit cache to ensure a clean environment.
Lastly, it executes all pre-commit hooks locally.

Example usage:

```go
mage runprecommit
```

**Returns:**

error: An error if any issue occurs at any of the three stages
of the process.

---

### RunTests()

```go
RunTests() error
```

RunTests executes all unit tests.

Example usage:

```go
mage runtests
```

**Returns:**

error: An error if any issue occurs while running the tests.

---

## Contributing

Pull requests are welcome. For major changes,
please open an issue first to discuss what
you would like to change.

---

## License

This project is licensed under the MIT
License - see the [LICENSE](https://github.com/facebookincubator/TTPForge/blob/main/LICENSE)
file for details.
28 changes: 16 additions & 12 deletions magefiles/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -2,44 +2,48 @@ module magefile

go 1.21

toolchain go1.21.1
toolchain go1.21.2

require (
github.com/l50/goutils/v2 v2.1.0
github.com/l50/goutils/v2 v2.1.3
github.com/magefile/mage v1.15.0
github.com/spf13/afero v1.9.5
github.com/spf13/afero v1.10.0
)

require (
dario.cat/mergo v1.0.0 // indirect
github.com/Microsoft/go-winio v0.6.1 // indirect
github.com/ProtonMail/go-crypto v0.0.0-20230717121422-5aa5874ade95 // indirect
github.com/ProtonMail/go-crypto v0.0.0-20230828082145-3c4c8a2d2371 // indirect
github.com/acomagu/bufpipe v1.0.4 // indirect
github.com/bitfield/script v0.22.0 // indirect
github.com/cloudflare/circl v1.3.3 // indirect
github.com/cyphar/filepath-securejoin v0.2.4 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/fatih/color v1.15.0 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-git/go-billy/v5 v5.4.1 // indirect
github.com/go-git/go-git/v5 v5.8.1 // indirect
github.com/go-git/go-billy/v5 v5.5.0 // indirect
github.com/go-git/go-git/v5 v5.9.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/itchyny/gojq v0.12.13 // indirect
github.com/itchyny/timefmt-go v0.1.5 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/otiai10/copy v1.12.0 // indirect
github.com/otiai10/copy v1.14.0 // indirect
github.com/pjbgf/sha1cd v0.3.0 // indirect
github.com/sergi/go-diff v1.3.1 // indirect
github.com/skeema/knownhosts v1.2.0 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
golang.org/x/crypto v0.12.0 // indirect
golang.org/x/crypto v0.13.0 // indirect
golang.org/x/mod v0.12.0 // indirect
golang.org/x/net v0.14.0 // indirect
golang.org/x/sys v0.11.0 // indirect
golang.org/x/text v0.12.0 // indirect
golang.org/x/tools v0.12.1-0.20230815132531-74c255bcf846 // indirect
golang.org/x/net v0.15.0 // indirect
golang.org/x/sync v0.3.0 // indirect
golang.org/x/sys v0.12.0 // indirect
golang.org/x/text v0.13.0 // indirect
golang.org/x/tools v0.13.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
mvdan.cc/sh/v3 v3.6.0 // indirect
)

replace github.com/spf13/afero => github.com/spf13/afero v1.2.1
Loading

0 comments on commit be4b0a5

Please sign in to comment.