oras-project · aviral26 · Aug 19, 2021 · Aug 24, 2021 · Sep 9, 2020 · Aug 24, 2021
diff --git a/.github/workflows/oras-release.yml b/.github/workflows/oras-release.yml
@@ -0,0 +1,59 @@
+name: Release oras-project/registry container image
+
+on:
+  push:
+    tags:
+      - "v[0-9]+.[0-9]+.[0-9]+-alpha"
+
+jobs:
+  publish:
+    name: Build and publish container image
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+    env:
+      REGISTRY: ghcr.io
+      REPOSITORY: ${{ format('{0}/registry', github.repository_owner) }}
+      DOCKER_BUILDTAGS: "include_oss include_gcs"
+      CGO_ENABLED: 1
+      GO111MODULE: "auto"
+      GOPATH: ${{ github.workspace }}
+      GOOS: linux
+      COMMIT_RANGE: ${{ github.event_name == 'pull_request' && format('{0}..{1}',github.event.pull_request.base.sha, github.event.pull_request.head.sha) || format('{0}..{1}', github.event.before, github.event.after) }}
+
+    steps:
+      - name: Get git tag
+        id: get_git_tag
+        run: echo ::set-output name=git_tag::${GITHUB_REF#refs/tags/}
+
+      - name: Check out source code
+        if: ${{ success() }}
+        uses: actions/checkout@v2
+        with:
+          ref: ${{ steps.get_git_tag.outputs.git_tag }}
+
+      - name: Set docker image tag
+        env:
+          GIT_TAG: ${{ steps.get_git_tag.outputs.git_tag }}
+        id: get_image_tag
+        run: echo ::set-output name=docker_tag::${GIT_TAG}
+
+      - name: Login to GitHub Container Registry
+        uses: docker/login-action@v1
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Build and push
+        if: ${{ success() }}
+        uses: docker/build-push-action@v2
+        with:
+          context: .
+          file: ./Dockerfile
+          platforms: linux/amd64
+          push: true
+          tags: |
+            ${{ env.REGISTRY }}/${{ env.REPOSITORY }}:${{ steps.get_image_tag.outputs.docker_tag }}
+            ${{ env.REGISTRY }}/${{ env.REPOSITORY }}:latest
diff --git a/README.md b/README.md
@@ -1,73 +1,160 @@
 # Distribution
 
-The toolset to pack, ship, store, and deliver content.
-
-This repository's main product is the Open Source Registry implementation
-for storing and distributing container images using the
-[OCI Distribution Specification](https://github.com/opencontainers/distribution-spec).
-The goal of this project is to provide a simple, secure, and scalable base
-for building a large scale registry solution or running a simple private registry.
-It is a core library for many registry operators including Docker Hub, GitHub Container Registry,
-GitLab Container Registry and DigitalOcean Container Registry, as well as the CNCF Harbor
-Project, and VMware Harbor Registry.
-
-<img src="/distribution-logo.svg" width="200px" />
-
-[![Build Status](https://github.com/distribution/distribution/workflows/CI/badge.svg?branch=main&event=push)](https://github.com/distribution/distribution/actions?query=workflow%3ACI)
-[![GoDoc](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white&style=flat-square)](https://pkg.go.dev/github.com/distribution/distribution)
-[![License: Apache-2.0](https://img.shields.io/badge/License-Apache--2.0-blue.svg)](LICENSE)
-[![codecov](https://codecov.io/gh/distribution/distribution/branch/main/graph/badge.svg)](https://codecov.io/gh/distribution/distribution)
-[![FOSSA Status](https://app.fossa.com/api/projects/custom%2B162%2Fgithub.com%2Fdistribution%2Fdistribution.svg?type=shield)](https://app.fossa.com/projects/custom%2B162%2Fgithub.com%2Fdistribution%2Fdistribution?ref=badge_shield)
-[![OCI Conformance](https://github.com/distribution/distribution/workflows/conformance/badge.svg)](https://github.com/distribution/distribution/actions?query=workflow%3Aconformance)
-
-This repository contains the following components:
-
-|**Component**       |Description                                                                                                                                                                                         |
-|--------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| **registry**       | An implementation of the [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec).                                                                                                 |
-| **libraries**      | A rich set of libraries for interacting with distribution components. Please see [godoc](https://pkg.go.dev/github.com/distribution/distribution) for details. **Note**: The interfaces for these libraries are **unstable**. |
-| **documentation**  | Docker's full documentation set is available at [docs.docker.com](https://docs.docker.com). This repository [contains the subset](docs/) related just to the registry.                                                                                                                                          |
-
-### How does this integrate with Docker, containerd, and other OCI client?
-
-Clients implement against the OCI specification and communicate with the
-registry using HTTP. This project contains a client implementation which
-is currently in use by Docker, however, it is deprecated for the
-[implementation in containerd](https://github.com/containerd/containerd/tree/master/remotes/docker)
-and will not support new features.
-
-### What are the long term goals of the Distribution project?
-
-The _Distribution_ project has the further long term goal of providing a
-secure tool chain for distributing content. The specifications, APIs and tools
-should be as useful with Docker as they are without.
-
-Our goal is to design a professional grade and extensible content distribution
-system that allow users to:
-
-* Enjoy an efficient, secured and reliable way to store, manage, package and
-  exchange content
-* Hack/roll their own on top of healthy open-source components
-* Implement their own home made solution through good specs, and solid
-  extensions mechanism.
-
-## Contribution
-
-Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to contribute
-issues, fixes, and patches to this project. If you are contributing code, see
-the instructions for [building a development environment](BUILDING.md).
-
-## Communication
-
-For async communication and long running discussions please use issues and pull requests on the github repo.
-This will be the best place to discuss design and implementation.
-
-For sync communication we have a #distribution channel in the [CNCF Slack](https://slack.cncf.io/)
-that everyone is welcome to join and chat about development.
-
-## Licenses
-
-The distribution codebase is released under the [Apache 2.0 license](LICENSE).
-The README.md file, and files in the "docs" folder are licensed under the
-Creative Commons Attribution 4.0 International License. You may obtain a
-copy of the license, titled CC-BY-4.0, at http://creativecommons.org/licenses/by/4.0/.
+This fork of [distribution/distribution](distribution-distribution) provides
+an experimental implementation of [reference types](reference-types).
+
+Features supported:
+
+- :heavy_check_mark: PUT ORAS Artifact Manifest
+- :heavy_check_mark: GET ORAS Artifact Manifest
+- :heavy_check_mark: LIST referrers
+  - [ ] Pagination support
+- [ ] Garbage Collection of reference types
+
+To power the `/referrers` API, the implementation creates and uses an index.
+See [referrers.md](docs/referrers.md) for details.
+
+## Usage - Push, Discover, Pull
+
+The following steps illustrate how ORAS artifacts can be stored and retrieved
+from a registry. The artifact in this example is a Notary V2
+[signature](signature).
+
+### Prerequisites
+
+- Local registry prototype instance
+- [docker-generate](https://github.com/shizhMSFT/docker-generate)
+- [nv2](https://github.com/notaryproject/nv2)
+- `curl`
+- `jq`
+
+### Push an image to your registry
+
+```shell
+# Initialize local registry variables
+regIp="127.0.0.1" && \
+  regPort="5000" && \
+  registry="$regIp:$regPort" && \
+  repo="busybox" && \
+  tag="latest" && \
+  image="$repo:$tag" && \
+  reference="$registry/$image"
+
+# Pull an image from docker hub and push to local registry
+docker pull $image && \
+  docker tag $image $reference && \
+  docker push $reference
+```
+
+### Generate image manifest and sign it
+
+```shell
+# Generate self-signed certificates
+openssl req \
+  -x509 \
+  -sha256 \
+  -nodes \
+  -newkey rsa:2048 \
+  -days 365 \
+  -subj "/CN=$regIp/O=example inc/C=IN/ST=Haryana/L=Gurgaon" \
+  -addext "subjectAltName=IP:$regIp" \
+  -keyout example.key \
+  -out example.crt
+
+# Generate image manifest
+manifestFile="manifest-to-sign.json" && \
+  docker generate manifest $image > $manifestFile
+
+# Sign manifest
+signatureFile="manifest-signature.jwt" && \
+  nv2 sign --method x509 \
+    -k example.key \
+    -c example.crt \
+    -r $reference \
+    -o $signatureFile \
+    file:$manifestFile
+```
+
+### Obtain manifest and signature digests
+
+```shell
+manifestDigest="sha256:$(sha256sum $manifestFile | cut -d " " -f 1)" && \
+  signatureDigest="sha256:$(sha256sum $signatureFile | cut -d " " -f 1)"
+```
+
+### Create an Artifact file referencing the manifest that was signed and its signature as blob
+
+```shell
+artifactFile="artifact.json" && \
+  artifactMediaType="application/vnd.cncf.oras.artifact.manifest.v1+json" && \
+  artifactType="application/vnd.cncf.notary.v2" && \
+  signatureMediaType="application/vnd.cncf.notary.signature.v2+jwt" && \
+  signatureFileSize=`wc -c < $signatureFile` && \
+  manifestMediaType="$(cat $manifestFile | jq -r '.mediaType')" && \
+  manifestFileSize=`wc -c < $manifestFile`
+
+cat <<EOF > $artifactFile
+{
+  "mediaType": "$artifactMediaType",
+  "artifactType": "$artifactType",
+  "blobs": [
+    {
+      "mediaType": "$signatureMediaType",
+      "digest": "$signatureDigest",
+      "size": $signatureFileSize
+    }
+  ],
+  "subject": {
+      "mediaType": "$manifestMediaType",
+      "digest": "$manifestDigest",
+      "size": $manifestFileSize
+  }
+}
+EOF
+```
+
+### Obtain artifact digest
+
+```shell
+artifactDigest="sha256:$(sha256sum $artifactFile | cut -d " " -f 1)"
+```
+
+### Push signature and artifact
+
+```shell
+# Initiate blob upload and obtain PUT location
+blobPutLocation=`curl -I -X POST -s http://$registry/v2/$repo/blobs/uploads/ | grep "Location: " | sed -e "s/Location: //;s/$/\&digest=$signatureDigest/;s/\r//"`
+
+# Push signature blob
+curl -X PUT -H "Content-Type: application/octet-stream" --data-binary @"$signatureFile" $blobPutLocation
+
+# Push artifact
+curl -X PUT --data-binary @"$artifactFile" -H "Content-Type: $artifactMediaType" "http://$registry/v2/$repo/manifests/$artifactDigest"
+```
+
+### List referrers
+
+```shell
+# Retrieve referrers
+curl -s "http://$registry/oras/artifacts/v1/$repo/manifests/$manifestDigest/referrers?artifactType=$artifactType" | jq
+```
+
+### Verify signature
+
+```shell
+# Retrieve signature
+artifactDigest=`curl -s "http://$registry/oras/artifacts/v1/$repo/manifests/$manifestDigest/referrers?artifactType=$artifactType" | jq -r '.references[0].digest'` && \
+  signatureDigest=`curl -s "http://$registry/oras/artifacts/v1/$repo/manifests/$artifactDigest" | jq -r '.blobs[0].digest'` && \
+  retrievedSignatureFile="retrieved-signature.json" && \
+  curl -s http://$registry/v2/$repo/blobs/$signatureDigest > $retrievedSignatureFile
+
+# Verify signature
+nv2 verify \
+  -f $retrievedSignatureFile \
+  -c example.crt \
+  file:$manifestFile
+```
+
+[distribution-distribution]: https://github.com/distribution/distribution
+[reference-types]: https://github.com/oras-project/artifacts-spec
+[signature]: https://github.com/notaryproject/nv2/tree/prototype-2/docs/nv2
diff --git a/docs/referrers.md b/docs/referrers.md
@@ -0,0 +1,95 @@
+[[__TOC__]]
+
+# ORAS Artifacts Distribution
+
+This document describes an experimental prototype that implements the
+[ORAS Artifact Manifest](https://github.com/oras-project/artifacts-spec) spec.
+
+## Implementation
+
+To power the [/referrers](https://github.com/oras-project/artifacts-spec/blob/main/manifest-referrers-api.md) API, the
+referrers of a manifest are indexed in the repository store. The following example illustrates the creation of this
+index.
+
+The `nginx:v1` image is already persisted:
+
+- repository: `nginx`
+- digest: `sha256:111ma2d22ae5ef400769fa51c84717264cd1520ac8d93dc071374c1be49a111m`
+- tag: `v1.0`
+
+The repository store layout is represented as:
+
+```bash
+<root>
+└── v2
+    └── repositories
+        └── nginx
+            └── _manifests
+                └── revisions
+                    └── sha256
+                        └── 111ma2d22ae5ef400769fa51c84717264cd1520ac8d93dc071374c1be49a111m
+                            └── link
+```
+
+Push a signature as blob and an ORAS Artifact that contains a blobs property referencing the signature, with the
+following properties:
+
+- digest: `sha256:222ibbf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cb222i`
+- `subjectManifest` digest: `sha256:111ma2d22ae5ef400769fa51c84717264cd1520ac8d93dc071374c1be49a111m`
+- `artifactType`: `application/vnd.example.artifact`
+
+On `PUT`, the artifact appears as a manifest revision. Additionally, an index entry is created under
+the subject to facilitate a lookup to the referrer. The index path where the entry is added is
+`/ref/<artifactType>`, as shown below.
+
+```
+<root>
+└── v2
+    └── repositories
+        └── nginx
+            └── _manifests
+                └── revisions
+                    └── sha256
+                        ├── 111ma2d22ae5ef400769fa51c84717264cd1520ac8d93dc071374c1be49a111m
+                        │   ├── link
+                        │   └── ref
+                        │       └── digest(application/vnd.example.artifact)
+                        │           └── sha256
+                        │               └── 222ibbf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cb222i
+                        │                   └── link
+                        └── 222ibbf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cb222i
+                            └── link
+```
+
+Push another ORAS artifact with the following properties:
+
+- digest: `sha256:333ic0c33ebc4a74a0a554c86ac2b28ddf3454a5ad9cf90ea8cea9f9e75c333i`
+- `subjectManifest` digest: `sha256:111ma2d22ae5ef400769fa51c84717264cd1520ac8d93dc071374c1be49a111m`
+- `artifactType`: `application/vnd.another.example.artifact`
+
+This results in an addition to the index as shown below.
+
+```
+<root>
+└── v2
+    └── repositories
+        └── nginx
+            └── _manifests
+                └── revisions
+                    └── sha256
+                        ├── 111ma2d22ae5ef400769fa51c84717264cd1520ac8d93dc071374c1be49a111m
+                        │   ├── link
+                        │   └── ref
+                        │       ├── digest(application/vnd.example.artifact)
+                        │       │   └── sha256
+                        │       │       └── 222ibbf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cb222i
+                        │       │           └── link
+                        │       └── digest(application/vnd.another.example.artifact)
+                        │           └── sha256
+                        │               └── 333ic0c33ebc4a74a0a554c86ac2b28ddf3454a5ad9cf90ea8cea9f9e75c333i
+                        │                   └── link
+                        ├── 222ibbf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cb222i
+                        │   └── link
+                        └── 333ic0c33ebc4a74a0a554c86ac2b28ddf3454a5ad9cf90ea8cea9f9e75c333i
+                            └── link
+```
diff --git a/go.mod b/go.mod
@@ -30,6 +30,7 @@ require (
 	github.com/ncw/swift v1.0.47
 	github.com/opencontainers/go-digest v1.0.0
 	github.com/opencontainers/image-spec v1.0.1
+	github.com/oras-project/artifacts-spec v0.0.0-20210830204616-bd9edb3a9b43
 	github.com/satori/go.uuid v1.2.0 // indirect
 	github.com/sirupsen/logrus v1.8.1
 	github.com/spf13/cobra v0.0.3