codecheck.Rmd

---
output:
  pdf_document:
    toc: false
    includes:
       in_header: codecheck-preamble.sty
    latex_engine: xelatex
---

```{r rsetup,eval=TRUE,include=FALSE}
require("codecheck")
require("knitr")
require("rprojroot")
require("yaml")
require("xtable")
require("tibble")
require("readr")
options(width = 60)
opts_chunk$set(cache=FALSE)

root = find_root("codecheck.yml")
```

```{r codecheck_logo, echo=FALSE,results='asis'}
latex_codecheck_logo()
```

```{r manifest, eval=TRUE, include=FALSE}
metadata = codecheck_metadata(root)
manifest = metadata$manifest

dest_dir = file.path(root, "codecheck", "outputs")
## Create the outputs directory if missing
if ( !dir.exists(dest_dir) ) {
  dir.create(dest_dir)
}

manifest_df = copy_manifest_files(root, metadata,
                                  dest_dir, keep_full_path = FALSE,
                                  overwrite = FALSE
                                  )
```

---
title: `r paste("CODECHECK certificate", metadata$certificate)`
subtitle: `r codecheck:::as_latex_url(metadata$report)`
---

```{r summary_metadata, echo=FALSE, results='asis'}
latex_summary_of_metadata(metadata)
```

```{r summary_manifest, echo=FALSE, results='asis'}
latex_summary_of_manifest(metadata, manifest_df, root)
```

# Summary

This code was (not) straightforward to codecheck.
[... ADD MORE INTERESTING FINDINGS HERE..]

\clearpage

# CODECHECKER notes

- instruct people to create the data folder
- renv doesn't install all packages successfully, but those can be fixed
  by running install.packages("x") manually on the ones that do not work.
  	- nlme package did not install with renv
	- ggspatial failed
	- Error installing package 'KernSmooth' on Mac OS
	- sf also failed. At that point, nothing afterwards is installed.
	- installing manually means that those will be the most recent versions, and
	  maybe not those in the lockfile.
- fig3_time_heat_tos.png is not created by the scripts, but this is actually Figure 3 in the paper. It seems to be that the first column of `fig3_time_heat_tos.png` ended up at the last column in `fig3_time_heat_tos_revised.png`. Apart from that, figure `fig3_time_heat_tos_revised.png` (which we reproduced) is identical to Figure 3 in the table
    - could be related to [this](https://stackoverflow.com/questions/70222688/round-date-function-returns-floor-date-instead-of-rounded-date) (round_date function changing behaviour) But not sure.
- table1 revisions and table2 revisions are not created by the scripts
- are the above actually used in the paper? This needs checking.
- csl and bib docs are referred to in the markdown, but they do not exist
  on the repository. csl has to be deleted for the markdown to knit.
- the rmarkdown knits to pdf but there are no instructions on how to make
  pdf knit, so it should probably knit it to html.
- the lorenz plot is too big the pdf, it goes off the end of the page. This would
  be fixed with using html format too.
- table 1 and 2 are identical to the paper. Table 1 is however build from two csv files, so the table is not identical to the one file that is linked in the report

This took 30 minutes to complete on my laptop.

## Recommendations

I suggest to the authors to consider the following suggestions for their next publication or workflow:

- ...

(document here if any of the suggestions were taken up by the authors in the meantime - do not remove any, keep track of contributions via feedback)

\clearpage

# Manifest files

```{r, echo=FALSE, results="asis", width=100}
for(i in c(1:nrow(manifest_df))) {
  path <- manifest_df[i,"dest"]
  if(stringr::str_ends(path, "(png|pdf|jpg|jpeg)")) {
    # include graphics with knitr has to happen at top level in chunks, see https://github.com/yihui/knitr/issues/1260
    # see also https://stackoverflow.com/questions/51268623/insert-images-using-knitrinclude-graphics-in-a-for-loop
    # knitr::include_graphics(manifest_df[1, "dest"], error = TRUE)
    # Therefore, generate Markdown code here:
    cat("## ", basename(path), "\n\n")
    cat("**Comment:** ", manifest_df[i,"comment"], "\n\n")
    cat(paste0("![", "Test ", "](", path, ")\n"))
  } else if (stringr::str_ends(path, "(Rout|txt)")) {
    cat("## ", basename(path), "\n\n")
    cat("\\scriptsize \n\n", "```txt\n")
    cat(readLines(path), sep = "\n")
    cat("\n\n``` \n\n", "\\normalsize \n\n")
  } else if (stringr::str_ends(path, "csv")) {
    cat("## ", basename(path), "\n\n", "Summary statistics of tabular data:", "\n\n")
    cat("\\scriptsize \n\n", "```txt\n")
    print(skimr::skim(read.csv(path)))
    cat("\n\n``` \n\n", "\\normalsize \n\n")
  } else {
    cat("## ", basename(path), "\n\n")
    cat("Cannot inlcude output file as figure.")
  }
  
  cat("\\clearpage \n\n")
}
```

\clearpage

## Acknowledgements

I would like to thank Dr Bhatt and his team for promptly answering any
queries I had with this reproduction.  CODECHECK is financially
supported by the Mozilla foundation.

# Citing this document

```{r, results='asis',echo=FALSE}
cite_certificate(metadata)
```

# About CODECHECK

This certificate confirms that the codechecker could independently
reproduce the results of a computational analysis given the data and
code from a third party.  A CODECHECK does not check whether the
original computation analysis is correct.  However, as all materials
required for the reproduction are freely available by following the
links in this document, the reader can then study for themselves the
code and data.


# About this document

This document was created using [R Markdown](https://rmarkdown.rstudio.com/) using the [`codecheck`](https://github.com/codecheckers/codecheck) R package.
`make codecheck.pdf` will regenerate the report file.

```{r}
sessionInfo()
```

```{r, include=FALSE, eval=FALSE}
# render this document in RStudio
rmarkdown::render("codecheck.Rmd", output_format = "pdf_document") 
```