Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve animint2pages by automatically taking screenshot of the animint #131

Merged
merged 47 commits into from
Oct 9, 2024
Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
73d540f
Chromote session nit
siddhesh195 Jun 12, 2024
a3e9ae5
Update DESCRIPTION
siddhesh195 Jun 18, 2024
c0e6f23
Update tests.yaml
siddhesh195 Jun 18, 2024
492fd17
Update z_pages.R
siddhesh195 Jun 19, 2024
a466f94
Update z_pages.R
siddhesh195 Jun 21, 2024
fde4ea0
corrected variable name
siddhesh195 Jun 21, 2024
d3617dc
changed from captureScreenshot() to screenshot()
siddhesh195 Jun 23, 2024
de2a97f
added new test to check screenshot in ghpages
siddhesh195 Jun 24, 2024
348cd0e
corrected hardcoded repo owner name to actual repo owner during the test
siddhesh195 Jun 24, 2024
6872e8d
corrected hardcoded repo name to actual repo name during the test
siddhesh195 Jun 24, 2024
bfc1b89
test if gh-pages branch exist
siddhesh195 Jun 24, 2024
a358ad3
saved screenshot to local directory
siddhesh195 Jun 24, 2024
07596ac
Add file name screenshot.png to the path to query
siddhesh195 Jun 24, 2024
fc2bdca
Remove try..catch
siddhesh195 Jun 24, 2024
04d28e2
Add gh to suggests and tests.yaml
siddhesh195 Jun 24, 2024
439dd85
Load library gh
siddhesh195 Jun 24, 2024
de1e12c
code for server pointing to temp directory
siddhesh195 Jun 29, 2024
80c2da3
added magick in suggests
siddhesh195 Jun 29, 2024
1ec415b
added magick in suggests
siddhesh195 Jun 30, 2024
c1f5560
testing on github actions
siddhesh195 Jul 1, 2024
12941ab
Use random port Ids to solve mac error
siddhesh195 Jul 1, 2024
2653f9b
added cropping feature
siddhesh195 Jul 3, 2024
5a2523f
start common server and copy new files to it's directory
siddhesh195 Jul 4, 2024
1f834d1
removed cropping feature for debugging
siddhesh195 Jul 4, 2024
1bf1937
pass server directory into animint2dir
siddhesh195 Jul 4, 2024
baabd4f
fixed server error, updated version, added NEWS
siddhesh195 Jul 6, 2024
dec0650
corrected name of the function
siddhesh195 Jul 10, 2024
8241ea8
removed duplicate lines and whitespaces
siddhesh195 Jul 12, 2024
7569a7c
Merge branch 'master' into screenshot-animint2pages
siddhesh195 Aug 29, 2024
de42e29
removed empty line
siddhesh195 Aug 29, 2024
82c998c
removed empty line
siddhesh195 Aug 29, 2024
20bd120
screenshot excludes widgets
Aug 29, 2024
fc23ea2
animint2pages example add title, source, color
Aug 29, 2024
75bf982
added empty line and space
siddhesh195 Aug 29, 2024
c6e28c7
Merge branch 'master' into screenshot-animint2pages
siddhesh195 Aug 31, 2024
3b98e9c
test passes without persistent server
tdhock Sep 1, 2024
c7dca84
Remove empty lines
siddhesh195 Sep 14, 2024
b12a42f
Refactor to reuse start_server
siddhesh195 Sep 15, 2024
445cbc4
Delete persistent server
siddhesh195 Sep 15, 2024
2603e3b
Stop the server after capturing screensjot
siddhesh195 Sep 17, 2024
0d88a28
Merge branch 'master' into screenshot-animint2pages
siddhesh195 Sep 17, 2024
9375dca
Increment version
siddhesh195 Sep 18, 2024
87e3324
Merge branch 'screenshot-animint2pages' of https://github.com/animint…
siddhesh195 Sep 18, 2024
1b81662
Delete stop_binary, check that package is loaded before taking screen…
siddhesh195 Sep 19, 2024
1f7e933
Fix test error
siddhesh195 Sep 19, 2024
f336ee4
removed additional arguments
siddhesh195 Sep 29, 2024
3338cf5
Rename stop_server to stop_servr
siddhesh195 Oct 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- uses: r-lib/actions/setup-r@v2
- uses: r-lib/actions/setup-r-dependencies@v2
with:
extra-packages: digest, RJSONIO, gtable, plyr, reshape2, scales, knitr
extra-packages: digest, RJSONIO, gtable, plyr, reshape2, scales, knitr,chromote, gh,magick


- name: install package
Expand Down
7 changes: 5 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: animint2
Title: Animated Interactive Grammar of Graphics
Version: 2024.3.12
Version: 2024.7.5
URL: https://animint.github.io/animint2/
BugReports: https://github.com/animint/animint2/issues
Authors@R: c(
Expand Down Expand Up @@ -100,7 +100,10 @@ Suggests:
nlme,
rpart,
svglite,
ggplot2
ggplot2,
chromote,
gh,
magick
Remotes: ropensci/RSelenium@v1.7.4
License: GPL-3
Encoding: UTF-8
Expand Down
4 changes: 4 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# Changes in version 2024.7.5 (PR#131)

- Added functionality to capture screenshot in animint2pages

# Changes in version 2024.3.12 (PR#119)

- Add validation checks for duplicate args passed geom and aes
Expand Down
16 changes: 12 additions & 4 deletions R/z_animint.R
Original file line number Diff line number Diff line change
Expand Up @@ -248,23 +248,31 @@ storeLayer <- function(meta, g, g.data.varied){
#' @example inst/examples/animint2dir.R
animint2dir <- function(plot.list, out.dir = NULL,
json.file = "plot.json", open.browser = interactive(),
css.file = "") {
css.file = "",persistent_server=FALSE, ...) {
if(is.null(out.dir)){
out.dir <- tempfile()
}
animint.js <- file.path(out.dir, "animint.js")
if(dir.exists(out.dir) && !file.exists(animint.js)){
if(dir.exists(out.dir) && !file.exists(animint.js) && persistent_server==FALSE){
stop(animint.js, " does not exist, so not removing out.dir. If you really want to save your animint in out.dir, then please remove that directory entirely")
}
unlink(out.dir, recursive=TRUE)
if(persistent_server==FALSE){
unlink(out.dir, recursive=TRUE)
}

## Check plot.list for errors
checkPlotList(plot.list)

## Store meta-data in this environment, so we can alter state in the
## lower-level functions.
meta <- newEnvironment()
meta$selector.types <- plot.list$selector.types
dir.create(out.dir,showWarnings=FALSE)
if(persistent_server==FALSE){

tdhock marked this conversation as resolved.
Show resolved Hide resolved
dir.create(out.dir,showWarnings=FALSE)

}

meta$out.dir <- out.dir
## First, copy html/js/json files to out.dir.
src.dir <- system.file("htmljs",package="animint2")
Expand Down
43 changes: 41 additions & 2 deletions R/z_pages.R
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

#' Publish a list of ggplots as interactive visualizations on a GitHub repository
#'
#' This function takes a named list of ggplots, generates interactive animations,
Expand Down Expand Up @@ -35,12 +36,17 @@
#' }
#'
#' @export
animint2pages <- function(plot.list, github_repo, commit_message = "Commit from animint2pages", private = FALSE, required_opts = c("title","source"), ...) {


tdhock marked this conversation as resolved.
Show resolved Hide resolved

animint2pages <- function(plot.list, github_repo, commit_message = "Commit from animint2pages", private = FALSE, required_opts = c("title","source"),server=NULL, ...) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @siddhesh195 , can you provide more details on how the param server works?

Copy link
Contributor Author

@siddhesh195 siddhesh195 Jul 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Faye-yufan server parameter is an object of persistent server which should be created in every test file having animint2pages as per current implementation if we want a persistent server. If server object is not passed during every call of animint2pages a new server with random available port is created. But if you see gh-pages test where I initialize server in the beginning or the code block that runs in animint2pages if no server object is passed, you will notice that I create a new R session for the server. This is because directly using servr API did not work properly for some reason to handle dynamic html even though it says it is in non blocking daemon mode. But having a separate R session makes it complex to terminate the server using code, so it just keeps running until that R environment is manually closed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Faye-yufan server parameter is an object of persistent server which should be created in every test file having animint2pages and it remains running when started. This avoids the need to create a new server in every call of animint2pages. But if you see gh-pages test where I initialize server in the beginning, you will notice that I create a new R session for the server. This is because directly using servr API did not work properly for some reason to handle dynamic html even though it says it is in non blocking daemon mode. But having a separate R session makes it complex to terminate the server, so it just keeps running until that R environment is closed.

if the server object is passed in aminint2pages, we need to tell animint2dir to use the same directory that the server is already pointing to, for that to work I had to make some changes to animint2dir function too.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add space after comma: , server=NULL

also please add docs for new server argument


for(opt in required_opts){
if(!opt %in% names(plot.list)){
stop(sprintf("plot.list does not contain option named %s, which is required by animint2pages", opt))
}
}

# Check for required packages
for(pkg in c("gert", "gh")){
if (!requireNamespace(pkg)) {
Expand All @@ -49,7 +55,38 @@ animint2pages <- function(plot.list, github_repo, commit_message = "Commit from
}
# Generate plot files
res <- animint2dir(plot.list, open.browser = FALSE, ...)
# Select non-ignored files to post

portNum <- servr::random_port()
chrome.session <- chromote::ChromoteSession$new()


if (is.null(server)) {
res <- animint2dir(plot.list, open.browser = FALSE, ...)
portNum <- servr::random_port()
normDir <- normalizePath(res$out.dir, winslash = "/", mustWork = TRUE)
code = sprintf("servr::httd(dir='%s', port=%d)", normDir, portNum)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this functionality (starting a server using servr::httd) seems duplicated here, and when printing animint, and in tests_init. is there some way to combine the repeated logic into a helper function?

system2("Rscript", c("-e", shQuote(code)), wait = FALSE)
Sys.sleep(3)
url <- sprintf("http://localhost:%d", portNum)
chrome.session$Page$navigate(url)
}else{
res <- animint2dir(plot.list, open.browser = FALSE,out.dir=server$output.dir,persistent_server=TRUE, ...)
url <- sprintf("http://localhost:%d",8080)
chrome.session$Page$navigate(url)
}

screenshot_path <- file.path(res$out.dir, "Capture.PNG")

Sys.sleep(3)

# Capture screenshot
screenshot <- chrome.session$Page$captureScreenshot()
image_raw <- magick::image_read(jsonlite::base64_dec(screenshot$data))
image_trimmed <- magick::image_trim(image_raw)
magick::image_write(image_trimmed, screenshot_path)

chrome.session$close()

all_files <- Sys.glob(file.path(res$out.dir, "*"))
file_info <- file.info(all_files)
to_post <- all_files[!(file_info$size == 0 | grepl("~$", all_files))]
Expand All @@ -75,6 +112,7 @@ animint2pages <- function(plot.list, github_repo, commit_message = "Commit from
repo <- gert::git_clone(origin_url, local_clone)
}
viz_url <- paste0("https://", owner, ".github.io/", github_repo)

# check if repo has commit, if not, give it first commit, this can avoid error
has_commits <- FALSE
try(
Expand All @@ -92,6 +130,7 @@ animint2pages <- function(plot.list, github_repo, commit_message = "Commit from
manage_gh_pages(repo, to_post, local_clone, commit_message)
message(sprintf(
"Visualization will be available at %s\nDeployment via GitHub Pages may take a few minutes...", viz_url))

viz_owner_repo
}

Expand Down
35 changes: 31 additions & 4 deletions tests/testthat/test-compiler-ghpages.R
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
acontext("GitHub Pages")
library(gh)
server <- function() {
out.dir <- tempfile()

dir.create(out.dir, showWarnings=FALSE, recursive=TRUE)
portNum <- 8080
normDir <- normalizePath(out.dir, winslash = "/", mustWork = TRUE)
code = sprintf("servr::httd(dir='%s', port=%d)", normDir,portNum)
system2("Rscript", c("-e", shQuote(code)), wait = FALSE)
return(list(output.dir = normDir))
}

s<-server()
tdhock marked this conversation as resolved.
Show resolved Hide resolved
viz <- animint(
title="one to ten",
source="https://github.com/animint/animint2/tree/master/tests/testthat/test-compiler-ghpages.R",
Expand All @@ -10,23 +22,38 @@ test_that("error for viz with no title", {
viz.no.title <- viz
viz.no.title$title <- NULL
expect_error({
animint2pages(viz.no.title, "no-title")
animint2pages(viz.no.title, "no-title",server=s)
}, "plot.list does not contain option named title, which is required by animint2pages")
})

test_that("error for viz with no source", {
viz.no.source <- viz
viz.no.source$source <- NULL
expect_error({
animint2pages(viz.no.source, "no-source")
animint2pages(viz.no.source, "no-source",server=s)
}, "plot.list does not contain option named source, which is required by animint2pages")
})

test_that("animint2pages() returns owner/repo string", {
viz_owner_repo <- animint2pages(viz, github_repo = "animint2pages_test_repo")
viz_owner_repo <- animint2pages(viz, github_repo = "animint2pages_test_repo",server=s)
tdhock marked this conversation as resolved.
Show resolved Hide resolved
expect_is(viz_owner_repo, "character")
})

test_that("check if animint2pages() successfully uploads screenshot", {

viz_owner_repo <- animint2pages(viz, github_repo = "animint2pages_test_repo",server=s)
split_viz_owner_repo <- strsplit(viz_owner_repo, "/")[[1]]
repo_owner <- split_viz_owner_repo[1]
repo_name <- split_viz_owner_repo[2]

file_exists <- {
gh("GET /repos/:owner/:repo/contents/:path",
owner = repo_owner, repo = repo_name, path = "Capture.PNG",ref ="gh-pages")
TRUE # If the call succeeds, the file exists
}
expect_true(file_exists, info = "The screenshot should exist in the repository.")
})

test_that("animint2pages raises an error if no GitHub token is present", {
env.names <- c("GITHUB_PAT", "GITHUB_PAT_GITHUB_COM")
env.old <- Sys.getenv(env.names)
Expand All @@ -40,7 +67,7 @@ test_that("animint2pages raises an error if no GitHub token is present", {
file.copy(config.file, config.old, overwrite = TRUE)
cat("[credential]\n\tusername = FOO", file=config.file, append=TRUE)
expect_error({
animint2pages(viz, github_repo = "test_repo")
animint2pages(viz, github_repo = "test_repo",server=s)
}, "A GitHub token is required to create and push to a new repository. \nTo create a GitHub token, follow these steps:\n1. Go to https://github.com/settings/tokens/new?scopes=repo&description=animint2pages\n2. Confirm your password if prompted.\n3. Ensure that the 'repo' scope is checked.\n4. Click 'Generate token' at the bottom of the page.\n5. Copy the generated token.\nAfter creating the token, you can set it up in your R environment by running: \nSys.setenv(GITHUB_PAT=\"yourGithubPAT\") \ngert::git_config_global_set(\"user.name\", \"yourUserName\") \ngert::git_config_global_set(\"user.email\", \"yourEmail\") \n", fixed=TRUE)
do.call(Sys.setenv, as.list(env.old))
file.copy(config.old, config.file, overwrite = TRUE)
Expand Down
Loading