Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid shutdown race condition on service startup #369

Closed
wants to merge 177 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
177 commits
Select commit Hold shift + click to select a range
ed9629d
Delete Travis Continuous Integration
dinosaure May 5, 2020
00f8338
Update GitHub Actions script
dinosaure May 5, 2020
9a698a2
Support ocamlformat
dinosaure May 5, 2020
417c8a2
Use dune 2.0
dinosaure May 5, 2020
37888a6
New implementation of conduit (core library)
dinosaure May 5, 2020
087e8e0
New implementation of conduit-tls which provides a composition mechan…
dinosaure May 5, 2020
50efb13
New implementation of conduit-lwt with mirage-flow implementation and…
dinosaure May 5, 2020
8dcb7df
New implementation of conduit-lwt-unix with Lwt's channel, tls and ss…
dinosaure May 5, 2020
8c44f50
New implementation of conduit-mirage with tcp/ip and tls layers
dinosaure May 5, 2020
1ce0f9a
New implementation of conduit-async with reader/writer helpers, tcp/i…
dinosaure May 5, 2020
402b68e
Add a ping-pong server to test conduit-{lwt-unix,async}
dinosaure May 5, 2020
4cc2bcb
Update OPAM files
dinosaure May 5, 2020
60ec59c
TCP linger used by conduit-lwt-unix is bigger (io_buffer_size)
dinosaure May 22, 2020
8ccfc86
Add interface helper for functoria/mirage about protocols impl.
dinosaure May 22, 2020
c5636a1
Add helper to easily register a resolver in mirage
dinosaure May 22, 2020
142fd9d
ocamlformat.0.14.2 pass
dinosaure May 26, 2020
7c09a83
Avoid to parse docstrings by ocamlformat
dinosaure Jun 4, 2020
fc38b9f
Integration of @samoht's simplication into the core library
dinosaure Jun 4, 2020
626c9b8
Update conduit-async with the new core
dinosaure Jun 4, 2020
bb4ee90
Update conduit-lwt with the new core
dinosaure Jun 4, 2020
ca12602
Update conduit-lwt-unix with the new core
dinosaure Jun 4, 2020
1fb8fc2
Update conduit-mirage with the new core
dinosaure Jun 4, 2020
66dbbc0
Update conduit-tls with the new core
dinosaure Jun 4, 2020
44125b6
Update tests with the new core
dinosaure Jun 4, 2020
c84a6b4
('a, 'b * 'c) service to ('a, 'b, 'c) service
dinosaure Jun 4, 2020
8b412ba
ocamlformat pass
dinosaure Jun 4, 2020
bd3af38
Rename End_of_input to End_of_flow
dinosaure Jun 4, 2020
1e89348
Add Conduit.Service.{accept,close}
dinosaure Jun 4, 2020
036a62d
ocamlformat pass
dinosaure Jun 4, 2020
aeaadfb
Remove prefix module name Client and add documentation
dinosaure Jun 8, 2020
1c36d32
Re-order documentation and ocamlformat pass
dinosaure Jun 8, 2020
8e75c74
Fix error about internal thd type and add a comment about it
dinosaure Jun 8, 2020
5401506
Add documentation
dinosaure Jun 9, 2020
96a9399
Rename connect to resolve and provide Conduit.connect as a simple cal…
dinosaure Jun 9, 2020
3a16024
Fallback API update to tests
dinosaure Jun 9, 2020
6748a1d
Fallback mirage-flow tests into conduit
dinosaure Jun 9, 2020
9143e28
ocamlformat pass
dinosaure Jun 9, 2020
94b2b97
Add tests about resolvers
dinosaure Jun 9, 2020
5843ee9
Add alcotest as a dependency to test core library
dinosaure Jun 9, 2020
44a3ce5
Functorize tests over LWT and ASYNC
dinosaure Jun 16, 2020
775cbf5
Delete conduit-lwt-unix and provide conduit-lwt-{ssl,tls} with lwt.unix
dinosaure Jun 20, 2020
aa9966a
Create new package conduit-async-{tls,ssl} and integrate conduit-asyn…
dinosaure Jun 22, 2020
7367a0b
Update GitHub Action with the new layout of conduit
dinosaure Jun 22, 2020
cfa3821
Optmize the core of conduit to have a smaller overhead about projection
dinosaure Jun 22, 2020
d4549a2
conduit-lwt-unix: require tls 0.12.2+
hannesm Jun 23, 2020
3f64382
Add a proper benchmark
dinosaure Jun 24, 2020
1f6936f
Add conduit-mirage.flow sub-package
dinosaure Jun 24, 2020
4215901
Add Conduit_lwt.io_of_flow
dinosaure Jun 24, 2020
e214f5e
Move librairies in their own directory
samoht Jun 29, 2020
a4a22fe
Minor naming cleanup
samoht Jun 29, 2020
96b789d
Simplify the code organisation for lwt and async libs
samoht Jun 29, 2020
7a8fc1e
Rename master -> service
samoht Jun 29, 2020
414f165
Refactor tests a bit
samoht Jun 29, 2020
a302c32
Remove `module type CONDUIT`
samoht Jun 29, 2020
f7613ac
Rename resolv_conf to resolve
samoht Jun 29, 2020
10e3d63
Be consistent when using Server/Service: try to use Service everywhere
samoht Jun 29, 2020
6fc534b
conduit-async-{tls,ssl} require conduit-async
dinosaure Jun 29, 2020
9779d77
Re-export resolvers type to signature
dinosaure Jul 10, 2020
e586e05
Merge pull request #320 from hannesm/mirage-crypto-0-8
avsm Jul 24, 2020
ad082e7
Resolver from the DNS stack return a Mirage TCP endpoint
dinosaure Aug 14, 2020
9815aa0
Add mirage-time dependency into conduit-mirage
dinosaure Aug 26, 2020
a44d4d9
Add Service.equal to be able to prove the type of cfg/t/flow
dinosaure Aug 30, 2020
2f6912c
Extend the initialization of a Async TCP service
dinosaure Aug 30, 2020
9d83881
Add ?timeout option on the Conduit_*.serve function
dinosaure Aug 30, 2020
40e4330
Merge remote-tracking branch 'origin/master' into 3.0.0
dinosaure Sep 2, 2020
e25fce5
Apply dune build @fmt --auto-promote
dinosaure Sep 2, 2020
ee0fa7d
Fix compilation about the addition of ?timeout argument into Conduit_…
dinosaure Sep 2, 2020
9da9b21
Fix GitHub Action and handle only ubuntu
dinosaure Sep 2, 2020
4dffc53
Delay the initialisation of the server
dinosaure Sep 8, 2020
d975a1b
Add ke and bigstringaf as a dependency of conduit-mirage (@hannesm)
dinosaure Sep 10, 2020
6493c30
conduit-mirage requires at least dns-client.4.6.0
dinosaure Sep 10, 2020
056e94b
Refine dependencies of conduit-async packages (@anuragsoni)
dinosaure Sep 11, 2020
db46ef3
Use ocamlformat.0.15.0 now!
dinosaure Sep 29, 2020
5e0a0b4
Fix the documentation on the core library
dinosaure Sep 29, 2020
66fd34a
Merge pull request #311 from dinosaure/3.0.0
avsm Sep 29, 2020
4b1adb9
Add endpoint and replace it by [ `host ] Domain_name.t
dinosaure Sep 29, 2020
e8d7b17
Use Result's module of stdlib-shims package
dinosaure Sep 30, 2020
daafc52
Define our own Result.bind operator
dinosaure Sep 30, 2020
f4ab264
Add documentation on Conduit.Endpoint module
dinosaure Oct 1, 2020
befb1a3
Merge pull request #322 from dinosaure/endpoint
dinosaure Oct 2, 2020
8912d45
Add ipaddr as a dependency of conduit
dinosaure Oct 2, 2020
175ab18
conduit-lwt-tls: initialize the mirage-crypto-rng.lwt PRNG
hannesm Oct 4, 2020
e0565d0
Merge pull request #325 from hannesm/rng
dinosaure Oct 4, 2020
665e6ac
Prepare 3.0.0
dinosaure Oct 6, 2020
4603c4b
Add an ocamldoc's HOW-TO about Conduit
dinosaure Oct 8, 2020
e055911
Update the CHANGES.md to include the small HOW-TO about conduit
dinosaure Oct 8, 2020
8f4f340
Merge pull request #326 from mirage/release-3.0.0
dinosaure Oct 8, 2020
b64478c
Minor update on index.mld
dinosaure Oct 9, 2020
7b517f6
Add a simple HOW-TO which explain how to change a protocol with Conduit
dinosaure Oct 9, 2020
69030be
Move the other howto (which is more complete) as readme.mld
dinosaure Oct 9, 2020
6ae5a0e
Update CHANGES.md
dinosaure Oct 11, 2020
60711d5
Update .gitignore about some trials with conduit
dinosaure Oct 11, 2020
5b48125
Fix use of lwt_io + conduit-tls
dinosaure Oct 12, 2020
aecc3d8
Fix Conduit_lwt.TCP protocol from the recv-first behavior of Lwt_io
dinosaure Oct 12, 2020
5611ef7
Emit the opportunity to re-schedule to the user
dinosaure Oct 12, 2020
c63d602
Really read first behavior or re-schedule on Conduit_lwt.TCP
dinosaure Oct 12, 2020
bcd85b5
Add yield on IO impl. needed to re-scheduler when we get `Input 0/0 (…
dinosaure Oct 12, 2020
ad1bc31
Re-schedule when we got 0 when we tried to send something
dinosaure Oct 12, 2020
b51d6de
Merge pull request #328 from mirage/fix-lwt-io
dinosaure Oct 13, 2020
a915c61
Delete recv_first and check Lwt_unix.readable to re-schedule or not t…
dinosaure Oct 13, 2020
82da075
Some useful logs about conduit-tls
dinosaure Oct 13, 2020
7ff1e0d
Delete the re-scheduling on the TLS layer according the state of the …
dinosaure Oct 13, 2020
857f1e0
Delete logs on tests (too verbose)
dinosaure Oct 13, 2020
a0e1e0b
Add a way to destruct the flow to the Conduit_lwt_ssl Lwt_ssl.socket
dinosaure Oct 13, 2020
82692fc
Add documentation about [io_of_flow]
dinosaure Oct 13, 2020
4b9e8b0
Merge pull request #329 from dinosaure/clean-fix-lwt-io-flow
dinosaure Oct 14, 2020
331786d
Minor fixes and add a paragraph about MirageOS
dinosaure Oct 14, 2020
2d126da
Merge pull request #327 from mirage/howto
dinosaure Oct 14, 2020
4cd8090
Minor fixes about documentation
dinosaure Oct 15, 2020
850d19b
Update the README.md
dinosaure Oct 15, 2020
78c850a
Merge pull request #330 from mirage/doc
dinosaure Oct 15, 2020
9f46649
The readme.mld is less detailed and more comprehensible than before
dinosaure Oct 15, 2020
042ef80
Merge pull request #331 from mirage/README
dinosaure Oct 15, 2020
4718b18
howto: no need to manually initialize the rng (conduit_lwt_tls does t…
hannesm Oct 15, 2020
7d5d2f7
Merge pull request #332 from hannesm/no-rng-unix-initialize
dinosaure Oct 15, 2020
8430dcc
Lint OPAM files
dinosaure Oct 16, 2020
2109ba3
Use post tag to be able to install tests
dinosaure Oct 16, 2020
f5ec040
Merge pull request #333 from mirage/lint-opam
dinosaure Oct 16, 2020
2c2c911
Clean tests
dinosaure Oct 19, 2020
8273960
Lint OPAM files
dinosaure Oct 19, 2020
126ace1
Merge pull request #334 from mirage/clean-tests
dinosaure Oct 19, 2020
e877c5d
Missing mirage-crypto-rng dependency on conduit-async-tls package
dinosaure Oct 20, 2020
c60090d
Add bench target to Makefile & allow json bench output
gs0510 Oct 30, 2020
5897dd5
Pin bench opam package for github actions
gs0510 Oct 30, 2020
1ebbd61
Merge pull request #335 from gs0510/run_bench
dinosaure Nov 1, 2020
c72831c
Bind a protocol when we register a service to be able to abstract the…
dinosaure Nov 18, 2020
90c0ebc
Merge pull request #337 from mirage/without-protocol
samoht Nov 29, 2020
e4af12a
Remove module types duplication
samoht Nov 29, 2020
ef91c5b
Format comments
samoht Nov 29, 2020
f82baca
Remove labels and rename Service.service to Service.t
samoht Nov 29, 2020
965df68
Merge pull request #340 from samoht/intf
samoht Nov 29, 2020
b57f635
Merge pull request #341 from samoht/format-comments
samoht Nov 29, 2020
eab97d3
Merge pull request #342 from samoht/remove-labels
samoht Nov 29, 2020
2b59b5e
Simplify Conduit.repr
samoht Nov 29, 2020
3a9476c
Remove Comduit.pack and Conduit.unpack
samoht Nov 29, 2020
5b03c98
Add tests about type equality and Conduit.repr
dinosaure Nov 29, 2020
e3f87c0
Merge pull request #343 from samoht/simplify-repr
dinosaure Nov 29, 2020
f2cc071
Merge pull request #344 from samoht/remove-pack
dinosaure Nov 29, 2020
06057f4
Rename Service.close into Service.stop
samoht Nov 29, 2020
260f246
Rename Service.pack to Service.flow
samoht Nov 29, 2020
54f77d3
Conduit.connect: swap argument order to use a "t-first" convention
samoht Nov 29, 2020
d6606d9
Merge pull request #345 from samoht/stop
dinosaure Nov 30, 2020
3cc2aab
Merge pull request #347 from samoht/connect
dinosaure Nov 30, 2020
ae530e5
Merge branch 'master' into pack
dinosaure Nov 30, 2020
916c954
Fix tests about the renaming of Service.{close,stop}
dinosaure Nov 30, 2020
d4be9d4
Merge pull request #346 from samoht/pack
dinosaure Nov 30, 2020
1dd5602
Better not found error
dinosaure Nov 30, 2020
921ce80
Delete the useless scheduler type
dinosaure Nov 30, 2020
9d03e80
Add configuration function to help to make required value to initiali…
dinosaure Nov 30, 2020
3b5b923
Merge pull request #353 from mirage/configuration
dinosaure Dec 1, 2020
367486b
Merge pull request #354 from mirage/delete-scheduler-type
dinosaure Dec 1, 2020
2f546ca
Merge pull request #355 from mirage/better-not-found
dinosaure Dec 1, 2020
71c411f
core: expand aliases exposed in _intf file
craigfe Dec 1, 2020
f770ce0
specialise partial applications when possible
samoht Dec 1, 2020
7375b55
Merge pull request #359 from samoht/optim-partial-app
dinosaure Dec 2, 2020
0205b79
Remove unused Conduit_lwt_flow module
dinosaure Dec 2, 2020
9f2fa3e
Update behavior of the LWT UNIX TCP/IP stack to call only one time sy…
dinosaure Dec 2, 2020
3fa1343
Avoid the recursion on the Async TCP/IP stack to call only one time s…
dinosaure Dec 2, 2020
2bf766c
Avoid the case (`Input 0) which should never appear
dinosaure Dec 2, 2020
89f6cd8
Re-export module types within S under Conduit.*
craigfe Dec 2, 2020
444350a
Re-export Conduit.Refl constructor
craigfe Dec 2, 2020
f6c0824
Merge pull request #357 from CraigFe/expand-intf-aliases
dinosaure Dec 2, 2020
372bf50
Re-export refl to be usable outside of conduit
dinosaure Dec 2, 2020
fc0888c
Delete multiple defintion of refl
dinosaure Dec 2, 2020
0bbf77e
Merge pull request #360 from mirage/remove-conduit-lwt-flow
dinosaure Dec 2, 2020
da6e545
Merge pull request #362 from mirage/better-lwt-tcp-protocol
dinosaure Dec 2, 2020
838df81
Copy/paste tls.lwt with minor update and provide a thread-safe implem…
dinosaure Dec 2, 2020
edaef88
Update Conduit_{async,lwt}_tls.TCP with the new way to make a service
dinosaure Dec 2, 2020
8e7283b
Merge pull request #361 from mirage/re-export-refl
dinosaure Dec 2, 2020
4dcf5bd
Merge pull request #363 from mirage/tls-thread-safe-best
dinosaure Dec 2, 2020
6dd33d4
Add a test that Service.equal is reflexive
craigfe Dec 2, 2020
052a2c4
Gitignore opam/ when it is a symlink
craigfe Dec 2, 2020
c217cb3
Merge pull request #365 from CraigFe/test-service-equality
dinosaure Dec 2, 2020
eaf54be
Merge pull request #366 from CraigFe/ignore-opam-symlink
dinosaure Dec 2, 2020
9a544cb
Remove unused Conduit.S.resolvers type
craigfe Dec 2, 2020
194a067
Merge pull request #364 from CraigFe/unused-resolvers-type
dinosaure Dec 3, 2020
e427423
Avoid shutdown race condition on service startup
craigfe Dec 4, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 13 additions & 8 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,28 @@ jobs:
runs-on: ${{ matrix.operating-system }}
strategy:
matrix:
ocaml-version: [ '4.08.1', '4.09.0', '4.10.0' ]
operating-system: [macos-latest, ubuntu-latest, windows-latest]
ocaml-version: [ '4.08.1', '4.09.0', '4.10.0', '4.11.1' ]
operating-system: [ ubuntu-latest ]
steps:
- uses: actions/checkout@master
- uses: avsm/setup-ocaml@master
- uses: actions/checkout@v2
- uses: avsm/setup-ocaml@v1
with:
ocaml-version: ${{ matrix.ocaml-version }}
- name: Deps
run: |
opam pin add -n conduit.dev .
opam pin add -n conduit-lwt.dev .
opam pin add -n conduit-lwt-unix.dev .
opam pin add -n conduit-async.dev .
opam pin add -n conduit-mirage.dev .
opam depext -y conduit conduit-lwt conduit-lwt-unix conduit-async conduit-mirage
opam install --deps-only .
opam pin add -n conduit-tls.dev .
opam pin add -n conduit-async-tls.dev .
opam pin add -n conduit-async-ssl.dev .
opam pin add -n conduit-lwt-tls.dev .
opam pin add -n conduit-lwt-ssl.dev .
opam pin add -n conduit-bench.dev .
opam depext -y conduit conduit-tls conduit-lwt conduit-async conduit-mirage
opam install --deps-only -t conduit conduit-tls conduit-lwt conduit-lwt-tls conduit-lwt-ssl conduit-async conduit-async-tls conduit-async-ssl conduit-mirage conduit-bench
- name: Build
run: opam exec -- dune build
- name: Test
run: opam exec -- dune runtest
run: opam exec -- dune runtest --no-buffer --verbose -j 1
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
_build/
_opam/
_opam
sandbox/
.*.swp
*.install
.merlin
*~
8 changes: 8 additions & 0 deletions .ocamlformat
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
version = 0.15.0
break-infix = fit-or-vertical
parse-docstrings = true
indicate-multiline-delimiters=no
nested-match=align
sequence-style=separator
break-before-in=auto
if-then-else=keyword-first
20 changes: 0 additions & 20 deletions .travis.yml

This file was deleted.

9 changes: 9 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,12 @@
## v3.0.0 (2020-10-06)

* **breaking change** - New version of `conduit`. Most of the new API has a
description on the pull-request
[#311](https://github.com/mirage/ocaml-conduit/pull/311). Documentation is
updated as well with a small HOW-TO (available
[here](https://mirage.github.io/ocaml-conduit/conduit/howto.html)) which
describes the new API.

## v2.2.2 (2020-06-14)

* conduit-lwt-unix no longer calls Mirage_crypto_rng_unix.initialize, and is
Expand Down
5 changes: 4 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: build clean test doc
.PHONY: build clean test doc bench

build:
dune build
Expand All @@ -11,3 +11,6 @@ clean:

doc:
dune build @doc

bench:
@dune exec -- ./bench/cost.exe --json
58 changes: 34 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,47 @@
## conduit -- an OCaml network connection establishment library
## Conduit - a framework to abstract the protocol

[![Build Status](https://travis-ci.org/mirage/ocaml-conduit.svg?branch=master)](https://travis-ci.org/mirage/ocaml-conduit)
The `conduit` library is a simple framework to abstract a _protocol_. It
permits, for a protocol implementer, to get rid the responsibility of the
protocols choice (such as the TLS implementation). So, it provides a degree of
abstraction from precise protocol (like TLS or TCP/IP) libraries used - since
there are a variety of them.

The `conduit` library takes care of establishing and listening for
TCP and SSL/TLS connections for the Lwt and Async libraries.
### Documentation & Tutorials

The reason this library exists is to provide a degree of abstraction
from the precise SSL library used, since there are a variety of ways
to bind to a library (e.g. the C FFI, or the Ctypes library), as well
as well as which library is used (just OpenSSL for now).
The documentation is available [here][doc].

By default, OpenSSL is used as the preferred connection library, but
you can force the use of the pure OCaml TLS stack by setting the
environment variable `CONDUIT_TLS=native` when starting your program.
A simple HOW-TO to describe how to implement a ping-pong server/client and how
to upgrade them with TLS is available [here][howto]

The opam packages available are:
A more complete (but less easier to understand) document to describe Conduit is
available [here][readme].

- `conduit`: the main `Conduit` module
- `conduit-lwt`: the portable Lwt implementation
- `conduit-lwt-unix`: the Lwt/Unix implementation
- `conduit-async` the Jane Street Async implementation
- `conduit-mirage`: the MirageOS compatible implementation
### Distribution

### Debugging

Some of the `Lwt_unix`-based modules use a non-empty `CONDUIT_DEBUG`
environment variable to output debugging information to standard error.
Just set this variable when running the program to see what URIs
are being resolved to.
Conduit comes with several packages:
- `conduit`: the core library which has only 3 dependencies
- `conduit-tls`: a Conduit-compatible [ocaml-tls][ocaml-tls] implementation
- `conduit-lwt`: the library with [lwt][lwt], it provides a Conduit-compatible
Host's TCP/IP protocol
- `conduit-async`: the library with [async][async], it provides a
Conduit-compatible Host's TCP/IP protocol
- `conduit-lwt-{ssl,tls}` provides the Host's TCP/IP protocol with SSL (OpenSSL)
and TLS (`ocaml-tls`)
- `conduit-async-{ssl,tls}` provides the Host's TCP/IP protocol with SSL
(OpenSSL) and TLS (`ocaml-tls`)
- `conduit-mirage` a Conduit-compatible [mirage-tcpip][mirage-tcpip] protocol

### Further Informartion

* **API Docs:** http://docs.mirage.io/
* **API Docs:** http://mirage.github.io/ocaml-conduit/
* **WWW:** https://github.com/mirage/ocaml-conduit
* **E-mail:** <mirageos-devel@lists.xenproject.org>
* **Bugs:** https://github.com/mirage/ocaml-conduit/issues

[doc]: https://mirage.github.io/ocaml-conduit/conduit/index.html
[howto]: https://mirage.github.io/ocaml-conduit/conduit/howto.html
[readme]: https://mirage.github.io/ocaml-conduit/conduit/readme.html
[ocaml-tls]: https://github.com/mirleft/ocaml-tls
[lwt]: https://github.com/ocsigen/lwt
[async]: https://github.com/janestreet/async
[mirage-tcpip]: https://github.com/mirage/mirage-tcpip
23 changes: 0 additions & 23 deletions async/conduit_async.ml

This file was deleted.

23 changes: 0 additions & 23 deletions async/dune

This file was deleted.

65 changes: 0 additions & 65 deletions async/private_ssl_dummy.ml

This file was deleted.

Loading