Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Contributing and testing housekeeping #440

Closed
wants to merge 1 commit into from
Closed

Contributing and testing housekeeping #440

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,11 +87,11 @@ jobs:
if: ${{ matrix.lint }}

- name: Run tests
run: mix test --trace --include proxy
run: mix test --trace --include integration --include proxy
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's leave the tag on by default please. I'd much rather contributors see failing test when they run mix test and have to start some containers, rather than seeing everything green when there could be issues. Thanks.

if: ${{ !matrix.coverage }}

- name: Run tests with coverage
run: mix coveralls.github --include proxy
run: mix coveralls.github --include integration --include proxy
if: ${{ matrix.coverage }}

- name: Run Dialyzer
Expand Down
59 changes: 40 additions & 19 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Mint 🌱

[![Build status badge](https://travis-ci.org/elixir-mint/mint.svg?branch=master)](https://travis-ci.org/elixir-mint/mint)
[![CI badge](https://github.com/elixir-mint/mint/actions/workflows/main.yml/badge.svg)](https://github.com/elixir-mint/mint/actions/workflows/main.yml)
[![Documentation badge](https://img.shields.io/badge/Documentation-ff69b4)][documentation]
[![Hex.pm badge](https://img.shields.io/badge/Package%20on%20hex.pm-informational)](https://hex.pm/packages/mint)
[![Coverage status badge](https://coveralls.io/repos/github/elixir-mint/mint/badge.svg?branch=main)](https://coveralls.io/github/elixir-mint/mint?branch=main)
Expand All @@ -19,11 +19,15 @@ defp deps do
end
```

Then, run `$ mix deps.get`.
Then, run:

```shell
mix deps.get
```

## Usage

Mint is different from most Erlang and Elixir HTTP clients because it provides a *process-less architecture*. Instead, Mint is based on a functional and immutable data structure that represents an HTTP connection. This data structure wraps a TCP or SSL socket. This allows for more fine-tailored architectures where the developer is responsible for wrapping the connection struct, such as having one process handle multiple connections or having different kinds of processes handle connections. You can think of Mint as [`:gen_tcp`](https://erlang.org/doc/man/gen_tcp.html) and [`:ssl`](https://www.erlang.org/doc/man/ssl.html), but with an understanding of the HTTP/1.1 and HTTP/2 protocols.
Mint is different from most Erlang and Elixir HTTP clients because it provides a _process-less architecture_. Instead, Mint is based on a functional and immutable data structure that represents an HTTP connection. This data structure wraps a TCP or SSL socket. This allows for more fine-tailored architectures where the developer is responsible for wrapping the connection struct, such as having one process handle multiple connections or having different kinds of processes handle connections. You can think of Mint as [`:gen_tcp`](https://erlang.org/doc/man/gen_tcp.html) and [`:ssl`](https://www.erlang.org/doc/man/ssl.html), but with an understanding of the HTTP/1.1 and HTTP/2 protocols.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, let's leave the * please (here and in all the README) 🙃 Thank you.


Let's see an example of a basic interaction with Mint. First, we start a connection through `Mint.HTTP.connect/3`:

Expand All @@ -37,7 +41,7 @@ This transparently chooses between HTTP/1 and HTTP/2. Then, we can send requests
iex> {:ok, conn, request_ref} = Mint.HTTP.request(conn, "GET", "/", [], "")
```

The connection socket runs in [*active mode*](http://erlang.org/doc/man/inet.html#setopts-2) (with `active: :once`), which means that the user of the library needs to handle [TCP messages](http://erlang.org/doc/man/gen_tcp.html#connect-4) and [SSL messages](http://erlang.org/doc/man/ssl.html#id66002):
The connection socket runs in [_active mode_](http://erlang.org/doc/man/inet.html#setopts-2) (with `active: :once`), which means that the user of the library needs to handle [TCP messages](http://erlang.org/doc/man/gen_tcp.html#connect-4) and [SSL messages](http://erlang.org/doc/man/ssl.html#id66002):

```elixir
iex> flush()
Expand All @@ -47,7 +51,6 @@ iex> flush()

Users are not supposed to examine these messages. Instead, Mint provides a `stream/2` function that turns messages into HTTP responses. Mint streams responses back to the user in parts through response parts such as `:status`, `:headers`, `:data`, and `:done`.


```elixir
iex> {:ok, conn} = Mint.HTTP.connect(:https, "httpbin.org", 443)
iex> {:ok, conn, request_ref} = Mint.HTTP.request(conn, "GET", "/", [], "")
Expand All @@ -66,7 +69,7 @@ iex> receive do

In the example above, we get all the responses as a single SSL message, but that might not always be the case. This means that `Mint.HTTP.stream/2` might not always return responses.

The connection API is *stateless*, which means that you need to make sure to always save the connection that functions return:
The connection API is _stateless_, which means that you need to make sure to always save the connection that functions return:

```elixir
# Wrong ❌
Expand Down Expand Up @@ -106,29 +109,47 @@ Mint is a low-level client. If you need higher-level features such as connection

If you wish to contribute, check out the [issue list][issues] and let us know what you want to work on, so that we can discuss it and reduce duplicate work.

Tests are organized with tags. Integration tests that hit real websites over the internet are tagged with `:requires_internet_connection`. Proxy tests are tagged with `:proxy` and require that you run `DOCKER_USER="$UID:$GID" docker-compose up` from the Mint root directory in order to run (they are excluded by default when you run `$ mix test`). A few examples of running tests:
Tests are organized with tags. Both the `:integration` and `:proxy` tagged tests are disabled by default, but can be enabled selectively.

* `$ mix test` to run the test suite without caring about Docker and `docker-compose up`.
If you encounter `{:error, %Mint.TransportError{reason: :nxdomain}}` error during the integration test, this suggests your DNS-based adblocker (self-hosted or VPN) might be blocking social media sites used in the integration test cases.

* `$ mix test --exclude integration` to only run local tests (for example, you don't have an internet connection available).
Here are a few examples of running tests.

Running all but `:integration` and `:proxy` tests:

```shell
mix test
```

* `$ mix test --include proxy` to run all tests, including proxy tests.
Local tests (for example, you don't have an Internet connection available).

```shell
mix test --exclude integration
```

Normal tests, including proxy or integration related test.

```shell
DOCKER_USER="$UID:$GID" docker-compose up --detach # or podman-compose up --detach
mix test --include proxy
mix test --include integration --include proxy
```

## License

Copyright 2018 Eric Meadows-Jönsson and Andrea Leopardi

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
Comment on lines +142 to +144
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no need to de-indent this, thanks!


http://www.apache.org/licenses/LICENSE-2.0
> https://www.apache.org/licenses/LICENSE-2.0
Copy link
Contributor Author

@kianmeng kianmeng Jun 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Markdown will generate this as hyperlink by quoting a URL link.


Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

[castore]: https://github.com/elixir-mint/castore
[documentation]: https://hexdocs.pm/mint
Expand Down
4 changes: 2 additions & 2 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,12 @@ services:
- "8889:8888"

httpbin:
image: kennethreitz/httpbin:latest
image: docker.io/kennethreitz/httpbin:latest
ports:
- "8080:80"

caddyhttpbin:
image: caddy:2.6.4-alpine
image: docker.io/caddy:2.6.4-alpine
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue when using podman-compose:

Error: short-name "caddy:2.6.4-alpine" did not resolve to an alias and no unqualified-search registries are defined in "/etc/containers/registries.conf"

# In GitHub Actions we want to access the files created
# by Caddy. However, in Linux these end up being owned by root
# because the container by default runs using the root user
Expand Down
2 changes: 1 addition & 1 deletion test/docker/tinyproxy-auth/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM alpine:3.17.2
FROM alpine:3.20.1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't seemed to get Apline Docker image 3.17.2 to work:

fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/main/x86_64/APKINDEX.tar.gz                                                             
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/main: temporary error (try again later)                                     
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/community/x86_64/APKINDEX.tar.gz                                                        
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/community: temporary error (try again later)
ERROR: unable to select packages:                                    
  tinyproxy (no such package): 
    required by: world[tinyproxy]

RUN apk add --no-cache \
tinyproxy
Expand Down
2 changes: 1 addition & 1 deletion test/docker/tinyproxy/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM alpine:3.17.2
FROM alpine:3.20.1

RUN apk add --no-cache \
tinyproxy
Expand Down
6 changes: 4 additions & 2 deletions test/mint/http1/integration_test.exs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ defmodule Mint.HTTP1.IntegrationTest do

alias Mint.{TransportError, HTTP1, HttpBin}

@moduletag :integration

describe "local httpbin" do
test "200 response" do
assert {:ok, conn} = HTTP1.connect(:http, "localhost", 8080)
Expand Down Expand Up @@ -113,6 +115,8 @@ defmodule Mint.HTTP1.IntegrationTest do
end

describe "httpbin.org" do
@describetag :integration

test "keep alive" do
assert {:ok, conn} =
HTTP1.connect(:https, HttpBin.host(), HttpBin.https_port(),
Expand Down Expand Up @@ -196,8 +200,6 @@ defmodule Mint.HTTP1.IntegrationTest do
end

describe "badssl.com" do
@describetag :requires_internet_connection

@tag :capture_log
test "SSL with bad certificate" do
assert {:error, %TransportError{reason: reason}} =
Expand Down
2 changes: 1 addition & 1 deletion test/mint/http2/integration_test.exs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ defmodule HTTP2.IntegrationTest do
alias Mint.HTTP2
alias Mint.HttpBin

@moduletag :requires_internet_connection
@moduletag :integration
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using shorter :integration tag to follow the short name convention of :proxy. Also, I kept typing require_internet_connection without the s.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I much prefer the explicit requires_internet_connection. Can we revert this change please?


setup context do
transport_opts =
Expand Down
2 changes: 1 addition & 1 deletion test/mint/integration_test.exs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ defmodule Mint.IntegrationTest do

alias Mint.{TransportError, HTTP, HttpBin}

@moduletag :requires_internet_connection
@moduletag :integration

describe "nghttp2.org" do
test "SSL - select HTTP1" do
Expand Down
2 changes: 1 addition & 1 deletion test/mint/tunnel_proxy_test.exs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ defmodule Mint.TunnelProxyTest do
alias Mint.HttpBin

@moduletag :proxy
@moduletag :requires_internet_connection
@moduletag :integration

test "200 response - http://httpbin.org" do
# Ensure we only match relevant messages
Expand Down
2 changes: 1 addition & 1 deletion test/mint/unsafe_proxy_test.exs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ defmodule Mint.UnsafeProxyTest do
alias Mint.HttpBin

@moduletag :proxy
@moduletag :requires_internet_connection
@moduletag :integration

test "200 response - http://httpbin.org" do
assert {:ok, conn} =
Expand Down
2 changes: 1 addition & 1 deletion test/test_helper.exs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
ExUnit.start(exclude: :proxy)
ExUnit.start(exclude: [:integration, :proxy])
Application.ensure_all_started(:ssl)
Copy link
Contributor Author

@kianmeng kianmeng Jun 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excludes the :integration and :proxy tagged tests by default, as it should work by default when running mix test.

Logger.configure(level: :info)

Expand Down
Loading