Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

207 test tarball runner + tweak rpc client logging #242

Merged
merged 19 commits into from
Aug 2, 2023
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
fe2b671
rpc-client: use MonadLogger instead of tracing
jberthold Jul 26, 2023
ee8c478
Cancel request in flight when rpc-client is interrupted (^C)
jberthold Jul 26, 2023
515e619
attempt to add a foundry bug report as a regression test, fails on kr…
jberthold Jul 25, 2023
4acc805
WIP making the llvm compilation work (semi-manually)
jberthold Jul 31, 2023
b84642b
add tarball test to integration tests
jberthold Jul 31, 2023
f46ba0b
fix prior (directory) tests
jberthold Jul 31, 2023
1f74305
Unpack and use all rpc_* directories in a tarball
jberthold Jul 31, 2023
dcfdf7d
create missing directories (tarball does not contain dir entries)
jberthold Jul 31, 2023
4ba0ab2
fix nix deps
goodlyrottenapple Jul 31, 2023
70df5b3
print raw response in debug mode
jberthold Aug 1, 2023
49482c4
DEBUG: run rpc-client with debug logging for integration test
jberthold Aug 1, 2023
ca4b856
Retry TCP connection before failing when the server is not ready
jberthold Aug 1, 2023
2b8e8fe
fml
jberthold Aug 1, 2023
c36cccb
replace tarball by a more up2date version
jberthold Aug 1, 2023
29968b1
Revert "DEBUG: run rpc-client with debug logging for integration test"
jberthold Aug 1, 2023
d3c3b81
reduce sleep time in runDirectoryTest (rpc-client will retry)
jberthold Aug 1, 2023
bde3cf9
DEBUG: run rpc-client with debug logging for integration test
jberthold Aug 1, 2023
4c78344
try integration test without debug again
jberthold Aug 1, 2023
a2c52f2
Merge branch 'main' into 207-test-tarball-runner_+_tweak-rpc-client-l…
jberthold Aug 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 135 additions & 0 deletions flake.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 6 additions & 1 deletion flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,18 @@
haskell-nix.follows = "haskell-backend/haskell-nix";
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that we are testing with a 5.6 version of K here... but upgrading seems to cause issues with the index-state being too new so I left that for a follow-up.

nixpkgs.follows = "haskell-backend/haskell-nix/nixpkgs-unstable";

blockchain-k-plugin.url = "github:runtimeverification/blockchain-k-plugin/da834be67f6c0aff11140ddfc0b04561494c14b8";
blockchain-k-plugin.inputs.flake-utils.follows = "k-framework/flake-utils";
blockchain-k-plugin.inputs.nixpkgs.follows = "k-framework/nixpkgs";

flake-compat = {
url = "github:edolstra/flake-compat";
flake = false;
};
};

outputs =
{ self, nixpkgs, haskell-nix, k-framework, haskell-backend, ... }@inputs:
{ self, nixpkgs, haskell-nix, k-framework, haskell-backend, blockchain-k-plugin, ... }@inputs:
let
inherit (nixpkgs) lib;
perSystem = lib.genAttrs nixpkgs.lib.systems.flakeExposed;
Expand Down Expand Up @@ -214,6 +218,7 @@
packages."hs-backend-booster:exe:kore-rpc-booster";
rpc-client = packages."hs-backend-booster:exe:rpc-client";
inherit (k-framework.packages.${system}) k;
blockchain-k-plugin = blockchain-k-plugin.defaultPackage.${system};
};
});

Expand Down
1 change: 1 addition & 0 deletions package.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ executables:
- extra
- filepath
- hs-backend-booster
- monad-logger
- network
- network-run
- optparse-applicative
Expand Down
6 changes: 4 additions & 2 deletions test/rpc-integration/default.nix
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{ stdenv, coreutils, lib, kore-rpc-booster, rpc-client, git, k }:
{ stdenv, coreutils, lib, kore-rpc-booster, rpc-client, git, k, blockchain-k-plugin, openssl, procps }:

let
mkIntegrationTest =
Expand All @@ -16,7 +16,8 @@ let
export ${f}
'') buildFlags}
export SERVER=${kore-rpc-booster}/bin/kore-rpc-booster
export CLIENT=${rpc-client}/bin/rpc-client
export CLIENT="${rpc-client}/bin/rpc-client -l debug"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran with debug logging to analyse the M1 failure but it is flaky... passed on the 2nd run after truncating some output on the first run.
Hopefully this won't become a problem later if/when we merge this.

export PLUGIN_DIR=${blockchain-k-plugin}

patchShebangs runDirectoryTest.sh
./runDirectoryTest.sh test-${name} --time ${
Expand Down Expand Up @@ -46,4 +47,5 @@ in {
simplify = mkIntegrationTest { name = "simplify"; };
get-model = mkIntegrationTest { name = "get-model"; };
issue212 = mkIntegrationTest { name = "issue212"; };
foundry-bug-report.tar.gz = mkIntegrationTest { name = "foundry-bug-report.tar.gz"; nativeBuildInputs = [ k openssl procps ]; };
}
48 changes: 48 additions & 0 deletions test/rpc-integration/resources/foundry-bug-report.tar.gz.kompile
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
set -eux

SCRIPT_DIR=$(dirname $0)
PLUGIN_DIR=${PLUGIN_DIR:-""}
NIX_LIBS=${NIX_LIBS:-""}


if [ -z "$PLUGIN_DIR" ]; then
echo "PLUGIN_DIR required to link in a crypto plugin dependency"
exit 1
else
for lib in libff libcryptopp libsecp256k1; do
LIBFILE=$(find ${PLUGIN_DIR} -name "${lib}.a" | head -1)
[ -z "$LIBFILE" ] && (echo "[Error] Unable to locate ${lib}.a"; exit 1)
PLUGIN_LIBS+="$LIBFILE "
PLUGIN_INCLUDE+="-I$(dirname $LIBFILE)/../include "
done
#PLUGIN_CPP=$(find ${PLUGIN_DIR}/plugin-c -name "*.cpp")
PLUGIN_CPP="${PLUGIN_DIR}/include/plugin-c/blake2.cpp ${PLUGIN_DIR}/include/plugin-c/crypto.cpp ${PLUGIN_DIR}/include/plugin-c/plugin_util.cpp"
fi

NAME=$(basename ${0%.tar.gz.kompile})
NAMETGZ=$(basename ${0%.kompile})

# unpack definition.kore and llvm_definition from tarball
# into a directory named after the tarball
cd $SCRIPT_DIR
mkdir -p $NAME
tar xzf ../test-$NAMETGZ -C $NAME definition.kore llvm_definition/

# provide definition
cp $NAME/definition.kore ${NAMETGZ}.kore

# kompile llvm-definition to interpreter

case "$OSTYPE" in
linux*) LPROCPS="-lprocps" ;;
*) LPROCPS="" ;;
esac

llvm-kompile $NAME/llvm_definition/definition.kore $NAME/llvm_definition/dt c -- \
-fPIC -std=c++17 -o interpreter \
$PLUGIN_LIBS $PLUGIN_INCLUDE $PLUGIN_CPP \
-lcrypto -lssl $LPROCPS
mv interpreter.* $NAMETGZ.dylib

# remove temporary artefacts
rm -r $NAME
39 changes: 22 additions & 17 deletions test/rpc-integration/runDirectoryTest.sh
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ dylib=resources/${dir#test-}.dylib
echo "Running tests from $dir with definition $kore"

# make sure directory and kore file exist
[ ! -d "./$dir" ] && exit 1
[ ! -e "./$dir" ] && exit 1
if [ ! -f "./$kore" ]; then
[ ! -f "./$kompile" ] && exit 2
cd resources
Expand All @@ -68,20 +68,25 @@ server_pid=$!
trap 'kill -2 ${server_pid}; popd' ERR EXIT
echo "Server PID ${server_pid}"

sleep 2
sleep 5

for test in $( ls $dir/state-*.{execute,send,simplify,add-module,get-model} 2>/dev/null ); do
tmp=${test#$dir/state-}
testname=${tmp%.*}
# determine send mode from suffix
mode=${test##*.}
printf "########## Test: %10s %20s\n" "$mode" "$testname #######"
if [ -f "$dir/params-${testname}.json" ]; then
params="--param-file $dir/params-${testname}.json"
else
params=""
fi
# call rpc-client
echo "$client $mode $test $params --expect $dir/response-${testname}.json $*"
$client $mode $test $params --expect $dir/response-${testname}.json $*
done
if [ -d $dir ]; then
for test in $( ls $dir/state-*.{execute,send,simplify,add-module,get-model} 2>/dev/null ); do
tmp=${test#$dir/state-}
testname=${tmp%.*}
# determine send mode from suffix
mode=${test##*.}
printf "########## Test: %10s %20s\n" "$mode" "$testname #######"
if [ -f "$dir/params-${testname}.json" ]; then
params="--param-file $dir/params-${testname}.json"
else
params=""
fi
# call rpc-client
echo "$client $mode $test $params --expect $dir/response-${testname}.json $*"
$client $mode $test $params --expect $dir/response-${testname}.json $*
done
else
echo "$dir is a file, running a tarball test"
$client run-tarball $dir
fi
Binary file not shown.
Loading