rmarkdown::render problem when called from a package












4














I've made a small package to reproduce the problem:



# example package
devtools::install_github("privefl/minipkg")

# example Rmd
rmd <- system.file("extdata", "Matrix.Rmd", package = "minipkg")
writeLines(readLines(rmd)) ## see content

# works fine
rmarkdown::render(
rmd,
"all",
envir = new.env(),
encoding = "UTF-8"
)

# !! does not work !!
minipkg::my_render(rmd)
minipkg::my_render ## see source code


I don't understand why the behaviour is different and how to fix this.



Edit: I know I can use Matrix::t(). My question is more "why do I need to use it in this particular case and not in all the other cases (such as calling rmarkdown::render() outside of a package)?".





Error



Quitting from lines 10-13 (Matrix.Rmd) 
Error in t.default(mat) : argument is not a matrix


Matrix.Rmd File



---
output: html_document
---
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

```{r}
library(Matrix)
mat <- rsparsematrix(10, 10, 0.1)
t(mat)
```


Console Output:



> # example package
> devtools::install_github("privefl/minipkg")
Downloading GitHub repo privefl/minipkg@master
✔ checking for file ‘/private/var/folders/md/03gdc4c14z18kbqwpfh4jdfc0000gr/T/RtmpKefs4h/remotes685793b9df4/privefl-minipkg-c02ae62/DESCRIPTION’ ...
─ preparing ‘minipkg’:
✔ checking DESCRIPTION meta-information ...
─ checking for LF line-endings in source and make files and shell scripts
─ checking for empty or unneeded directories
─ building ‘minipkg_0.1.0.tar.gz’

* installing *source* package ‘minipkg’ ...
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (minipkg)
> # example Rmd
> rmd <- system.file("extdata", "Matrix.Rmd", package = "minipkg")
> writeLines(readLines(rmd)) ## see content
---
output: html_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

```{r}
library(Matrix)
mat <- rsparsematrix(10, 10, 0.1)
t(mat)
```

> # works fine
> rmarkdown::render(
+ rmd,
+ "all",
+ envir = new.env(),
+ encoding = "UTF-8"
+ )


processing file: Matrix.Rmd
|............. | 20%
ordinary text without R code

|.......................... | 40%
label: setup (with options)
List of 1
$ include: logi FALSE

|....................................... | 60%
ordinary text without R code

|.................................................... | 80%
label: unnamed-chunk-1
|.................................................................| 100%
ordinary text without R code


output file: Matrix.knit.md

/usr/local/bin/pandoc +RTS -K512m -RTS Matrix.utf8.md --to html4 --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash+smart --output Matrix.html --email-obfuscation none --self-contained --standalone --section-divs --template /Library/Frameworks/R.framework/Versions/3.5/Resources/library/rmarkdown/rmd/h/default.html --no-highlight --variable highlightjs=1 --variable 'theme:bootstrap' --include-in-header /var/folders/md/03gdc4c14z18kbqwpfh4jdfc0000gr/T//RtmpKefs4h/rmarkdown-str68525040df1.html --mathjax --variable 'mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' --metadata pagetitle=Matrix.utf8.md

Output created: Matrix.html
> # !! does not work !!
> minipkg::my_render(rmd)


processing file: Matrix.Rmd
|............. | 20%
ordinary text without R code

|.......................... | 40%
label: setup (with options)
List of 1
$ include: logi FALSE

|....................................... | 60%
ordinary text without R code

|.................................................... | 80%
label: unnamed-chunk-1
Quitting from lines 10-13 (Matrix.Rmd)
Error in t.default(mat) : argument is not a matrix

> minipkg::my_render ## see source code
function (rmd)
{
rmarkdown::render(rmd, "all", envir = new.env(), encoding = "UTF-8")
}
<bytecode: 0x7f89c416c2a8>
<environment: namespace:minipkg>
>









share|improve this question





























    4














    I've made a small package to reproduce the problem:



    # example package
    devtools::install_github("privefl/minipkg")

    # example Rmd
    rmd <- system.file("extdata", "Matrix.Rmd", package = "minipkg")
    writeLines(readLines(rmd)) ## see content

    # works fine
    rmarkdown::render(
    rmd,
    "all",
    envir = new.env(),
    encoding = "UTF-8"
    )

    # !! does not work !!
    minipkg::my_render(rmd)
    minipkg::my_render ## see source code


    I don't understand why the behaviour is different and how to fix this.



    Edit: I know I can use Matrix::t(). My question is more "why do I need to use it in this particular case and not in all the other cases (such as calling rmarkdown::render() outside of a package)?".





    Error



    Quitting from lines 10-13 (Matrix.Rmd) 
    Error in t.default(mat) : argument is not a matrix


    Matrix.Rmd File



    ---
    output: html_document
    ---
    ```{r setup, include=FALSE}
    knitr::opts_chunk$set(echo = TRUE)
    ```

    ```{r}
    library(Matrix)
    mat <- rsparsematrix(10, 10, 0.1)
    t(mat)
    ```


    Console Output:



    > # example package
    > devtools::install_github("privefl/minipkg")
    Downloading GitHub repo privefl/minipkg@master
    ✔ checking for file ‘/private/var/folders/md/03gdc4c14z18kbqwpfh4jdfc0000gr/T/RtmpKefs4h/remotes685793b9df4/privefl-minipkg-c02ae62/DESCRIPTION’ ...
    ─ preparing ‘minipkg’:
    ✔ checking DESCRIPTION meta-information ...
    ─ checking for LF line-endings in source and make files and shell scripts
    ─ checking for empty or unneeded directories
    ─ building ‘minipkg_0.1.0.tar.gz’

    * installing *source* package ‘minipkg’ ...
    ** R
    ** inst
    ** byte-compile and prepare package for lazy loading
    ** help
    *** installing help indices
    ** building package indices
    ** testing if installed package can be loaded
    * DONE (minipkg)
    > # example Rmd
    > rmd <- system.file("extdata", "Matrix.Rmd", package = "minipkg")
    > writeLines(readLines(rmd)) ## see content
    ---
    output: html_document
    ---

    ```{r setup, include=FALSE}
    knitr::opts_chunk$set(echo = TRUE)
    ```

    ```{r}
    library(Matrix)
    mat <- rsparsematrix(10, 10, 0.1)
    t(mat)
    ```

    > # works fine
    > rmarkdown::render(
    + rmd,
    + "all",
    + envir = new.env(),
    + encoding = "UTF-8"
    + )


    processing file: Matrix.Rmd
    |............. | 20%
    ordinary text without R code

    |.......................... | 40%
    label: setup (with options)
    List of 1
    $ include: logi FALSE

    |....................................... | 60%
    ordinary text without R code

    |.................................................... | 80%
    label: unnamed-chunk-1
    |.................................................................| 100%
    ordinary text without R code


    output file: Matrix.knit.md

    /usr/local/bin/pandoc +RTS -K512m -RTS Matrix.utf8.md --to html4 --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash+smart --output Matrix.html --email-obfuscation none --self-contained --standalone --section-divs --template /Library/Frameworks/R.framework/Versions/3.5/Resources/library/rmarkdown/rmd/h/default.html --no-highlight --variable highlightjs=1 --variable 'theme:bootstrap' --include-in-header /var/folders/md/03gdc4c14z18kbqwpfh4jdfc0000gr/T//RtmpKefs4h/rmarkdown-str68525040df1.html --mathjax --variable 'mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' --metadata pagetitle=Matrix.utf8.md

    Output created: Matrix.html
    > # !! does not work !!
    > minipkg::my_render(rmd)


    processing file: Matrix.Rmd
    |............. | 20%
    ordinary text without R code

    |.......................... | 40%
    label: setup (with options)
    List of 1
    $ include: logi FALSE

    |....................................... | 60%
    ordinary text without R code

    |.................................................... | 80%
    label: unnamed-chunk-1
    Quitting from lines 10-13 (Matrix.Rmd)
    Error in t.default(mat) : argument is not a matrix

    > minipkg::my_render ## see source code
    function (rmd)
    {
    rmarkdown::render(rmd, "all", envir = new.env(), encoding = "UTF-8")
    }
    <bytecode: 0x7f89c416c2a8>
    <environment: namespace:minipkg>
    >









    share|improve this question



























      4












      4








      4







      I've made a small package to reproduce the problem:



      # example package
      devtools::install_github("privefl/minipkg")

      # example Rmd
      rmd <- system.file("extdata", "Matrix.Rmd", package = "minipkg")
      writeLines(readLines(rmd)) ## see content

      # works fine
      rmarkdown::render(
      rmd,
      "all",
      envir = new.env(),
      encoding = "UTF-8"
      )

      # !! does not work !!
      minipkg::my_render(rmd)
      minipkg::my_render ## see source code


      I don't understand why the behaviour is different and how to fix this.



      Edit: I know I can use Matrix::t(). My question is more "why do I need to use it in this particular case and not in all the other cases (such as calling rmarkdown::render() outside of a package)?".





      Error



      Quitting from lines 10-13 (Matrix.Rmd) 
      Error in t.default(mat) : argument is not a matrix


      Matrix.Rmd File



      ---
      output: html_document
      ---
      ```{r setup, include=FALSE}
      knitr::opts_chunk$set(echo = TRUE)
      ```

      ```{r}
      library(Matrix)
      mat <- rsparsematrix(10, 10, 0.1)
      t(mat)
      ```


      Console Output:



      > # example package
      > devtools::install_github("privefl/minipkg")
      Downloading GitHub repo privefl/minipkg@master
      ✔ checking for file ‘/private/var/folders/md/03gdc4c14z18kbqwpfh4jdfc0000gr/T/RtmpKefs4h/remotes685793b9df4/privefl-minipkg-c02ae62/DESCRIPTION’ ...
      ─ preparing ‘minipkg’:
      ✔ checking DESCRIPTION meta-information ...
      ─ checking for LF line-endings in source and make files and shell scripts
      ─ checking for empty or unneeded directories
      ─ building ‘minipkg_0.1.0.tar.gz’

      * installing *source* package ‘minipkg’ ...
      ** R
      ** inst
      ** byte-compile and prepare package for lazy loading
      ** help
      *** installing help indices
      ** building package indices
      ** testing if installed package can be loaded
      * DONE (minipkg)
      > # example Rmd
      > rmd <- system.file("extdata", "Matrix.Rmd", package = "minipkg")
      > writeLines(readLines(rmd)) ## see content
      ---
      output: html_document
      ---

      ```{r setup, include=FALSE}
      knitr::opts_chunk$set(echo = TRUE)
      ```

      ```{r}
      library(Matrix)
      mat <- rsparsematrix(10, 10, 0.1)
      t(mat)
      ```

      > # works fine
      > rmarkdown::render(
      + rmd,
      + "all",
      + envir = new.env(),
      + encoding = "UTF-8"
      + )


      processing file: Matrix.Rmd
      |............. | 20%
      ordinary text without R code

      |.......................... | 40%
      label: setup (with options)
      List of 1
      $ include: logi FALSE

      |....................................... | 60%
      ordinary text without R code

      |.................................................... | 80%
      label: unnamed-chunk-1
      |.................................................................| 100%
      ordinary text without R code


      output file: Matrix.knit.md

      /usr/local/bin/pandoc +RTS -K512m -RTS Matrix.utf8.md --to html4 --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash+smart --output Matrix.html --email-obfuscation none --self-contained --standalone --section-divs --template /Library/Frameworks/R.framework/Versions/3.5/Resources/library/rmarkdown/rmd/h/default.html --no-highlight --variable highlightjs=1 --variable 'theme:bootstrap' --include-in-header /var/folders/md/03gdc4c14z18kbqwpfh4jdfc0000gr/T//RtmpKefs4h/rmarkdown-str68525040df1.html --mathjax --variable 'mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' --metadata pagetitle=Matrix.utf8.md

      Output created: Matrix.html
      > # !! does not work !!
      > minipkg::my_render(rmd)


      processing file: Matrix.Rmd
      |............. | 20%
      ordinary text without R code

      |.......................... | 40%
      label: setup (with options)
      List of 1
      $ include: logi FALSE

      |....................................... | 60%
      ordinary text without R code

      |.................................................... | 80%
      label: unnamed-chunk-1
      Quitting from lines 10-13 (Matrix.Rmd)
      Error in t.default(mat) : argument is not a matrix

      > minipkg::my_render ## see source code
      function (rmd)
      {
      rmarkdown::render(rmd, "all", envir = new.env(), encoding = "UTF-8")
      }
      <bytecode: 0x7f89c416c2a8>
      <environment: namespace:minipkg>
      >









      share|improve this question















      I've made a small package to reproduce the problem:



      # example package
      devtools::install_github("privefl/minipkg")

      # example Rmd
      rmd <- system.file("extdata", "Matrix.Rmd", package = "minipkg")
      writeLines(readLines(rmd)) ## see content

      # works fine
      rmarkdown::render(
      rmd,
      "all",
      envir = new.env(),
      encoding = "UTF-8"
      )

      # !! does not work !!
      minipkg::my_render(rmd)
      minipkg::my_render ## see source code


      I don't understand why the behaviour is different and how to fix this.



      Edit: I know I can use Matrix::t(). My question is more "why do I need to use it in this particular case and not in all the other cases (such as calling rmarkdown::render() outside of a package)?".





      Error



      Quitting from lines 10-13 (Matrix.Rmd) 
      Error in t.default(mat) : argument is not a matrix


      Matrix.Rmd File



      ---
      output: html_document
      ---
      ```{r setup, include=FALSE}
      knitr::opts_chunk$set(echo = TRUE)
      ```

      ```{r}
      library(Matrix)
      mat <- rsparsematrix(10, 10, 0.1)
      t(mat)
      ```


      Console Output:



      > # example package
      > devtools::install_github("privefl/minipkg")
      Downloading GitHub repo privefl/minipkg@master
      ✔ checking for file ‘/private/var/folders/md/03gdc4c14z18kbqwpfh4jdfc0000gr/T/RtmpKefs4h/remotes685793b9df4/privefl-minipkg-c02ae62/DESCRIPTION’ ...
      ─ preparing ‘minipkg’:
      ✔ checking DESCRIPTION meta-information ...
      ─ checking for LF line-endings in source and make files and shell scripts
      ─ checking for empty or unneeded directories
      ─ building ‘minipkg_0.1.0.tar.gz’

      * installing *source* package ‘minipkg’ ...
      ** R
      ** inst
      ** byte-compile and prepare package for lazy loading
      ** help
      *** installing help indices
      ** building package indices
      ** testing if installed package can be loaded
      * DONE (minipkg)
      > # example Rmd
      > rmd <- system.file("extdata", "Matrix.Rmd", package = "minipkg")
      > writeLines(readLines(rmd)) ## see content
      ---
      output: html_document
      ---

      ```{r setup, include=FALSE}
      knitr::opts_chunk$set(echo = TRUE)
      ```

      ```{r}
      library(Matrix)
      mat <- rsparsematrix(10, 10, 0.1)
      t(mat)
      ```

      > # works fine
      > rmarkdown::render(
      + rmd,
      + "all",
      + envir = new.env(),
      + encoding = "UTF-8"
      + )


      processing file: Matrix.Rmd
      |............. | 20%
      ordinary text without R code

      |.......................... | 40%
      label: setup (with options)
      List of 1
      $ include: logi FALSE

      |....................................... | 60%
      ordinary text without R code

      |.................................................... | 80%
      label: unnamed-chunk-1
      |.................................................................| 100%
      ordinary text without R code


      output file: Matrix.knit.md

      /usr/local/bin/pandoc +RTS -K512m -RTS Matrix.utf8.md --to html4 --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash+smart --output Matrix.html --email-obfuscation none --self-contained --standalone --section-divs --template /Library/Frameworks/R.framework/Versions/3.5/Resources/library/rmarkdown/rmd/h/default.html --no-highlight --variable highlightjs=1 --variable 'theme:bootstrap' --include-in-header /var/folders/md/03gdc4c14z18kbqwpfh4jdfc0000gr/T//RtmpKefs4h/rmarkdown-str68525040df1.html --mathjax --variable 'mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' --metadata pagetitle=Matrix.utf8.md

      Output created: Matrix.html
      > # !! does not work !!
      > minipkg::my_render(rmd)


      processing file: Matrix.Rmd
      |............. | 20%
      ordinary text without R code

      |.......................... | 40%
      label: setup (with options)
      List of 1
      $ include: logi FALSE

      |....................................... | 60%
      ordinary text without R code

      |.................................................... | 80%
      label: unnamed-chunk-1
      Quitting from lines 10-13 (Matrix.Rmd)
      Error in t.default(mat) : argument is not a matrix

      > minipkg::my_render ## see source code
      function (rmd)
      {
      rmarkdown::render(rmd, "all", envir = new.env(), encoding = "UTF-8")
      }
      <bytecode: 0x7f89c416c2a8>
      <environment: namespace:minipkg>
      >






      r r-markdown






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 23 at 20:02

























      asked Nov 21 at 10:16









      F. Privé

      6,6952840




      6,6952840
























          1 Answer
          1






          active

          oldest

          votes


















          2





          +200









          How it works



          The problem is envir = new.env().
          What you need is envir = new.env(parent = globalenv()):



          devtools::install_github("privefl/minipkg")
          rmd <- system.file("extdata", "Matrix.Rmd", package = "minipkg")

          minipkg::my_render(rmd)
          # Fails

          f <- minipkg::my_render
          body(f) <- quote(rmarkdown::render(rmd, "all", envir = new.env(parent = globalenv()), encoding = "UTF-8"))

          ns <- getNamespace("minipkg")
          unlockBinding("my_render", ns)
          assign("my_render", f, envir = ns)

          minipkg::my_render(rmd)
          # Patched one works :)


          Why it works



          Look at the default arguments of new.env() to find that the default parent environment is parent.frame(). Note that from the console, this will be globalenv() and from within a package it will be this packages namespace (not the same as package environment!).



          You can get a package namespace with getNamespace("pkg"). It is the environment that contains all (also internal) objects of the package. The problem is that this environment is in a sense "disconnected" from the usual search / method lookup mechanic in R and so you won't find the necessary methods even though they are attached to search().



          Now chosing new.env(parent = globalenv()) sets the parent environment to be on the top of the search path and thus able to find all attached methods.



          Benchmarking different approaches



          These three approaches all produce proper html files:



          #' Render an Rmd file
          #' @param rmd Path of the R Markdown file to render.
          #' @export
          my_render <- function(rmd) {
          rmarkdown::render(
          rmd,
          "all",
          envir = new.env(parent = globalenv()),
          encoding = "UTF-8"
          )
          }

          #' Render an Rmd file
          #' @param rmd Path of the R Markdown file to render.
          #' @export
          my_render2 <- function(rmd) {
          cl <- parallel::makePSOCKcluster(1)
          on.exit(parallel::stopCluster(cl), add = TRUE)
          parallel::clusterExport(cl, "rmd", envir = environment())
          parallel::clusterEvalQ(cl, {
          rmarkdown::render(rmd, "all", encoding = "UTF-8")
          })[[1]]
          }

          #' Render an Rmd file
          #' @param rmd Path of the R Markdown file to render.
          #' @export
          my_render3 <- function(rmd) {
          system2(
          command = "R",
          args = c("-e", shQuote(sprintf("rmarkdown::render('%s', 'all', encoding = 'UTF-8')", gsub("\\", "/", normalizePath(rmd))))),
          wait = TRUE
          )
          }


          Now it's interesting to compare their speed:



          > microbenchmark::microbenchmark(my_render("inst/extdata/Matrix.Rmd"), my_render2("inst/extdata/Matrix.Rmd"), my_render3("inst/extdata/Matrix.Rmd"), times = 10L)

          [...]

          Unit: milliseconds
          expr min lq mean median uq max neval
          my_render("inst/extdata/Matrix.Rmd") 352.7927 410.604 656.5211 460.0608 560.3386 1836.452 10
          my_render2("inst/extdata/Matrix.Rmd") 1981.8844 2015.541 2163.1875 2118.0030 2307.2812 2407.027 10
          my_render3("inst/extdata/Matrix.Rmd") 2061.7076 2079.574 2152.0351 2138.9546 2181.1284 2377.623 10


          Conclusions





          • envir = new.env(globalenv()) is by far the fastest (almost 4x faster than the alternatives)

            I expect the overhead to be constant, so it should be irrelevant for larger Rmd files.

          • There is no discernible difference between spawning a new proces with system2 and using a parallel SOCK cluster with 1 node.






          share|improve this answer























          • Good job at finding the root of the problem. Yet, it feels wrong to add the global environment to the search path for the Rmd. Does this means that the Rmd has access to the current global env? I was thinking about using new.env(parent = parent.env(globalenv())). But this means that the Rmd has access to packages loaded before, right?
            – F. Privé
            Nov 23 at 22:49












          • @F.Privé That might work with already attached packages but new library() calls might modify the search path „above“. Also, source() calls modify globalenv() unless local = TRUE is specified. All in all it‘s probably safer to do system(„RScript -e rmarkdown::render(...)“) and fire up a fresh R session for the knitting. This is essentially what RStudio does when you click Knit.
            – AlexR
            Nov 24 at 5:40












          • I'm not a great fan of system calls inside packages. But using the "same idea" and executing it in a PSOCK cluster, it seems to work fine.
            – F. Privé
            Nov 24 at 6:15










          • See minipkg::my_render2().
            – F. Privé
            Nov 24 at 6:19










          • @F.Privé That‘s not much different, maybe a bit more overhead for setting up R in slave mode. Anyway, using a parallel package just hides the creation of the R process - still safer than playing with the envir parameter but I think system or system2 are more efficient and clearer as to why. Plus on linux a (default) FORK cluster will have a copy of globalenv(), so care must be taken to specify the cluster type.
            – AlexR
            Nov 24 at 6:19











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53409789%2frmarkdownrender-problem-when-called-from-a-package%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          2





          +200









          How it works



          The problem is envir = new.env().
          What you need is envir = new.env(parent = globalenv()):



          devtools::install_github("privefl/minipkg")
          rmd <- system.file("extdata", "Matrix.Rmd", package = "minipkg")

          minipkg::my_render(rmd)
          # Fails

          f <- minipkg::my_render
          body(f) <- quote(rmarkdown::render(rmd, "all", envir = new.env(parent = globalenv()), encoding = "UTF-8"))

          ns <- getNamespace("minipkg")
          unlockBinding("my_render", ns)
          assign("my_render", f, envir = ns)

          minipkg::my_render(rmd)
          # Patched one works :)


          Why it works



          Look at the default arguments of new.env() to find that the default parent environment is parent.frame(). Note that from the console, this will be globalenv() and from within a package it will be this packages namespace (not the same as package environment!).



          You can get a package namespace with getNamespace("pkg"). It is the environment that contains all (also internal) objects of the package. The problem is that this environment is in a sense "disconnected" from the usual search / method lookup mechanic in R and so you won't find the necessary methods even though they are attached to search().



          Now chosing new.env(parent = globalenv()) sets the parent environment to be on the top of the search path and thus able to find all attached methods.



          Benchmarking different approaches



          These three approaches all produce proper html files:



          #' Render an Rmd file
          #' @param rmd Path of the R Markdown file to render.
          #' @export
          my_render <- function(rmd) {
          rmarkdown::render(
          rmd,
          "all",
          envir = new.env(parent = globalenv()),
          encoding = "UTF-8"
          )
          }

          #' Render an Rmd file
          #' @param rmd Path of the R Markdown file to render.
          #' @export
          my_render2 <- function(rmd) {
          cl <- parallel::makePSOCKcluster(1)
          on.exit(parallel::stopCluster(cl), add = TRUE)
          parallel::clusterExport(cl, "rmd", envir = environment())
          parallel::clusterEvalQ(cl, {
          rmarkdown::render(rmd, "all", encoding = "UTF-8")
          })[[1]]
          }

          #' Render an Rmd file
          #' @param rmd Path of the R Markdown file to render.
          #' @export
          my_render3 <- function(rmd) {
          system2(
          command = "R",
          args = c("-e", shQuote(sprintf("rmarkdown::render('%s', 'all', encoding = 'UTF-8')", gsub("\\", "/", normalizePath(rmd))))),
          wait = TRUE
          )
          }


          Now it's interesting to compare their speed:



          > microbenchmark::microbenchmark(my_render("inst/extdata/Matrix.Rmd"), my_render2("inst/extdata/Matrix.Rmd"), my_render3("inst/extdata/Matrix.Rmd"), times = 10L)

          [...]

          Unit: milliseconds
          expr min lq mean median uq max neval
          my_render("inst/extdata/Matrix.Rmd") 352.7927 410.604 656.5211 460.0608 560.3386 1836.452 10
          my_render2("inst/extdata/Matrix.Rmd") 1981.8844 2015.541 2163.1875 2118.0030 2307.2812 2407.027 10
          my_render3("inst/extdata/Matrix.Rmd") 2061.7076 2079.574 2152.0351 2138.9546 2181.1284 2377.623 10


          Conclusions





          • envir = new.env(globalenv()) is by far the fastest (almost 4x faster than the alternatives)

            I expect the overhead to be constant, so it should be irrelevant for larger Rmd files.

          • There is no discernible difference between spawning a new proces with system2 and using a parallel SOCK cluster with 1 node.






          share|improve this answer























          • Good job at finding the root of the problem. Yet, it feels wrong to add the global environment to the search path for the Rmd. Does this means that the Rmd has access to the current global env? I was thinking about using new.env(parent = parent.env(globalenv())). But this means that the Rmd has access to packages loaded before, right?
            – F. Privé
            Nov 23 at 22:49












          • @F.Privé That might work with already attached packages but new library() calls might modify the search path „above“. Also, source() calls modify globalenv() unless local = TRUE is specified. All in all it‘s probably safer to do system(„RScript -e rmarkdown::render(...)“) and fire up a fresh R session for the knitting. This is essentially what RStudio does when you click Knit.
            – AlexR
            Nov 24 at 5:40












          • I'm not a great fan of system calls inside packages. But using the "same idea" and executing it in a PSOCK cluster, it seems to work fine.
            – F. Privé
            Nov 24 at 6:15










          • See minipkg::my_render2().
            – F. Privé
            Nov 24 at 6:19










          • @F.Privé That‘s not much different, maybe a bit more overhead for setting up R in slave mode. Anyway, using a parallel package just hides the creation of the R process - still safer than playing with the envir parameter but I think system or system2 are more efficient and clearer as to why. Plus on linux a (default) FORK cluster will have a copy of globalenv(), so care must be taken to specify the cluster type.
            – AlexR
            Nov 24 at 6:19
















          2





          +200









          How it works



          The problem is envir = new.env().
          What you need is envir = new.env(parent = globalenv()):



          devtools::install_github("privefl/minipkg")
          rmd <- system.file("extdata", "Matrix.Rmd", package = "minipkg")

          minipkg::my_render(rmd)
          # Fails

          f <- minipkg::my_render
          body(f) <- quote(rmarkdown::render(rmd, "all", envir = new.env(parent = globalenv()), encoding = "UTF-8"))

          ns <- getNamespace("minipkg")
          unlockBinding("my_render", ns)
          assign("my_render", f, envir = ns)

          minipkg::my_render(rmd)
          # Patched one works :)


          Why it works



          Look at the default arguments of new.env() to find that the default parent environment is parent.frame(). Note that from the console, this will be globalenv() and from within a package it will be this packages namespace (not the same as package environment!).



          You can get a package namespace with getNamespace("pkg"). It is the environment that contains all (also internal) objects of the package. The problem is that this environment is in a sense "disconnected" from the usual search / method lookup mechanic in R and so you won't find the necessary methods even though they are attached to search().



          Now chosing new.env(parent = globalenv()) sets the parent environment to be on the top of the search path and thus able to find all attached methods.



          Benchmarking different approaches



          These three approaches all produce proper html files:



          #' Render an Rmd file
          #' @param rmd Path of the R Markdown file to render.
          #' @export
          my_render <- function(rmd) {
          rmarkdown::render(
          rmd,
          "all",
          envir = new.env(parent = globalenv()),
          encoding = "UTF-8"
          )
          }

          #' Render an Rmd file
          #' @param rmd Path of the R Markdown file to render.
          #' @export
          my_render2 <- function(rmd) {
          cl <- parallel::makePSOCKcluster(1)
          on.exit(parallel::stopCluster(cl), add = TRUE)
          parallel::clusterExport(cl, "rmd", envir = environment())
          parallel::clusterEvalQ(cl, {
          rmarkdown::render(rmd, "all", encoding = "UTF-8")
          })[[1]]
          }

          #' Render an Rmd file
          #' @param rmd Path of the R Markdown file to render.
          #' @export
          my_render3 <- function(rmd) {
          system2(
          command = "R",
          args = c("-e", shQuote(sprintf("rmarkdown::render('%s', 'all', encoding = 'UTF-8')", gsub("\\", "/", normalizePath(rmd))))),
          wait = TRUE
          )
          }


          Now it's interesting to compare their speed:



          > microbenchmark::microbenchmark(my_render("inst/extdata/Matrix.Rmd"), my_render2("inst/extdata/Matrix.Rmd"), my_render3("inst/extdata/Matrix.Rmd"), times = 10L)

          [...]

          Unit: milliseconds
          expr min lq mean median uq max neval
          my_render("inst/extdata/Matrix.Rmd") 352.7927 410.604 656.5211 460.0608 560.3386 1836.452 10
          my_render2("inst/extdata/Matrix.Rmd") 1981.8844 2015.541 2163.1875 2118.0030 2307.2812 2407.027 10
          my_render3("inst/extdata/Matrix.Rmd") 2061.7076 2079.574 2152.0351 2138.9546 2181.1284 2377.623 10


          Conclusions





          • envir = new.env(globalenv()) is by far the fastest (almost 4x faster than the alternatives)

            I expect the overhead to be constant, so it should be irrelevant for larger Rmd files.

          • There is no discernible difference between spawning a new proces with system2 and using a parallel SOCK cluster with 1 node.






          share|improve this answer























          • Good job at finding the root of the problem. Yet, it feels wrong to add the global environment to the search path for the Rmd. Does this means that the Rmd has access to the current global env? I was thinking about using new.env(parent = parent.env(globalenv())). But this means that the Rmd has access to packages loaded before, right?
            – F. Privé
            Nov 23 at 22:49












          • @F.Privé That might work with already attached packages but new library() calls might modify the search path „above“. Also, source() calls modify globalenv() unless local = TRUE is specified. All in all it‘s probably safer to do system(„RScript -e rmarkdown::render(...)“) and fire up a fresh R session for the knitting. This is essentially what RStudio does when you click Knit.
            – AlexR
            Nov 24 at 5:40












          • I'm not a great fan of system calls inside packages. But using the "same idea" and executing it in a PSOCK cluster, it seems to work fine.
            – F. Privé
            Nov 24 at 6:15










          • See minipkg::my_render2().
            – F. Privé
            Nov 24 at 6:19










          • @F.Privé That‘s not much different, maybe a bit more overhead for setting up R in slave mode. Anyway, using a parallel package just hides the creation of the R process - still safer than playing with the envir parameter but I think system or system2 are more efficient and clearer as to why. Plus on linux a (default) FORK cluster will have a copy of globalenv(), so care must be taken to specify the cluster type.
            – AlexR
            Nov 24 at 6:19














          2





          +200







          2





          +200



          2




          +200




          How it works



          The problem is envir = new.env().
          What you need is envir = new.env(parent = globalenv()):



          devtools::install_github("privefl/minipkg")
          rmd <- system.file("extdata", "Matrix.Rmd", package = "minipkg")

          minipkg::my_render(rmd)
          # Fails

          f <- minipkg::my_render
          body(f) <- quote(rmarkdown::render(rmd, "all", envir = new.env(parent = globalenv()), encoding = "UTF-8"))

          ns <- getNamespace("minipkg")
          unlockBinding("my_render", ns)
          assign("my_render", f, envir = ns)

          minipkg::my_render(rmd)
          # Patched one works :)


          Why it works



          Look at the default arguments of new.env() to find that the default parent environment is parent.frame(). Note that from the console, this will be globalenv() and from within a package it will be this packages namespace (not the same as package environment!).



          You can get a package namespace with getNamespace("pkg"). It is the environment that contains all (also internal) objects of the package. The problem is that this environment is in a sense "disconnected" from the usual search / method lookup mechanic in R and so you won't find the necessary methods even though they are attached to search().



          Now chosing new.env(parent = globalenv()) sets the parent environment to be on the top of the search path and thus able to find all attached methods.



          Benchmarking different approaches



          These three approaches all produce proper html files:



          #' Render an Rmd file
          #' @param rmd Path of the R Markdown file to render.
          #' @export
          my_render <- function(rmd) {
          rmarkdown::render(
          rmd,
          "all",
          envir = new.env(parent = globalenv()),
          encoding = "UTF-8"
          )
          }

          #' Render an Rmd file
          #' @param rmd Path of the R Markdown file to render.
          #' @export
          my_render2 <- function(rmd) {
          cl <- parallel::makePSOCKcluster(1)
          on.exit(parallel::stopCluster(cl), add = TRUE)
          parallel::clusterExport(cl, "rmd", envir = environment())
          parallel::clusterEvalQ(cl, {
          rmarkdown::render(rmd, "all", encoding = "UTF-8")
          })[[1]]
          }

          #' Render an Rmd file
          #' @param rmd Path of the R Markdown file to render.
          #' @export
          my_render3 <- function(rmd) {
          system2(
          command = "R",
          args = c("-e", shQuote(sprintf("rmarkdown::render('%s', 'all', encoding = 'UTF-8')", gsub("\\", "/", normalizePath(rmd))))),
          wait = TRUE
          )
          }


          Now it's interesting to compare their speed:



          > microbenchmark::microbenchmark(my_render("inst/extdata/Matrix.Rmd"), my_render2("inst/extdata/Matrix.Rmd"), my_render3("inst/extdata/Matrix.Rmd"), times = 10L)

          [...]

          Unit: milliseconds
          expr min lq mean median uq max neval
          my_render("inst/extdata/Matrix.Rmd") 352.7927 410.604 656.5211 460.0608 560.3386 1836.452 10
          my_render2("inst/extdata/Matrix.Rmd") 1981.8844 2015.541 2163.1875 2118.0030 2307.2812 2407.027 10
          my_render3("inst/extdata/Matrix.Rmd") 2061.7076 2079.574 2152.0351 2138.9546 2181.1284 2377.623 10


          Conclusions





          • envir = new.env(globalenv()) is by far the fastest (almost 4x faster than the alternatives)

            I expect the overhead to be constant, so it should be irrelevant for larger Rmd files.

          • There is no discernible difference between spawning a new proces with system2 and using a parallel SOCK cluster with 1 node.






          share|improve this answer














          How it works



          The problem is envir = new.env().
          What you need is envir = new.env(parent = globalenv()):



          devtools::install_github("privefl/minipkg")
          rmd <- system.file("extdata", "Matrix.Rmd", package = "minipkg")

          minipkg::my_render(rmd)
          # Fails

          f <- minipkg::my_render
          body(f) <- quote(rmarkdown::render(rmd, "all", envir = new.env(parent = globalenv()), encoding = "UTF-8"))

          ns <- getNamespace("minipkg")
          unlockBinding("my_render", ns)
          assign("my_render", f, envir = ns)

          minipkg::my_render(rmd)
          # Patched one works :)


          Why it works



          Look at the default arguments of new.env() to find that the default parent environment is parent.frame(). Note that from the console, this will be globalenv() and from within a package it will be this packages namespace (not the same as package environment!).



          You can get a package namespace with getNamespace("pkg"). It is the environment that contains all (also internal) objects of the package. The problem is that this environment is in a sense "disconnected" from the usual search / method lookup mechanic in R and so you won't find the necessary methods even though they are attached to search().



          Now chosing new.env(parent = globalenv()) sets the parent environment to be on the top of the search path and thus able to find all attached methods.



          Benchmarking different approaches



          These three approaches all produce proper html files:



          #' Render an Rmd file
          #' @param rmd Path of the R Markdown file to render.
          #' @export
          my_render <- function(rmd) {
          rmarkdown::render(
          rmd,
          "all",
          envir = new.env(parent = globalenv()),
          encoding = "UTF-8"
          )
          }

          #' Render an Rmd file
          #' @param rmd Path of the R Markdown file to render.
          #' @export
          my_render2 <- function(rmd) {
          cl <- parallel::makePSOCKcluster(1)
          on.exit(parallel::stopCluster(cl), add = TRUE)
          parallel::clusterExport(cl, "rmd", envir = environment())
          parallel::clusterEvalQ(cl, {
          rmarkdown::render(rmd, "all", encoding = "UTF-8")
          })[[1]]
          }

          #' Render an Rmd file
          #' @param rmd Path of the R Markdown file to render.
          #' @export
          my_render3 <- function(rmd) {
          system2(
          command = "R",
          args = c("-e", shQuote(sprintf("rmarkdown::render('%s', 'all', encoding = 'UTF-8')", gsub("\\", "/", normalizePath(rmd))))),
          wait = TRUE
          )
          }


          Now it's interesting to compare their speed:



          > microbenchmark::microbenchmark(my_render("inst/extdata/Matrix.Rmd"), my_render2("inst/extdata/Matrix.Rmd"), my_render3("inst/extdata/Matrix.Rmd"), times = 10L)

          [...]

          Unit: milliseconds
          expr min lq mean median uq max neval
          my_render("inst/extdata/Matrix.Rmd") 352.7927 410.604 656.5211 460.0608 560.3386 1836.452 10
          my_render2("inst/extdata/Matrix.Rmd") 1981.8844 2015.541 2163.1875 2118.0030 2307.2812 2407.027 10
          my_render3("inst/extdata/Matrix.Rmd") 2061.7076 2079.574 2152.0351 2138.9546 2181.1284 2377.623 10


          Conclusions





          • envir = new.env(globalenv()) is by far the fastest (almost 4x faster than the alternatives)

            I expect the overhead to be constant, so it should be irrelevant for larger Rmd files.

          • There is no discernible difference between spawning a new proces with system2 and using a parallel SOCK cluster with 1 node.







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Nov 28 at 11:06

























          answered Nov 23 at 20:19









          AlexR

          1,961922




          1,961922












          • Good job at finding the root of the problem. Yet, it feels wrong to add the global environment to the search path for the Rmd. Does this means that the Rmd has access to the current global env? I was thinking about using new.env(parent = parent.env(globalenv())). But this means that the Rmd has access to packages loaded before, right?
            – F. Privé
            Nov 23 at 22:49












          • @F.Privé That might work with already attached packages but new library() calls might modify the search path „above“. Also, source() calls modify globalenv() unless local = TRUE is specified. All in all it‘s probably safer to do system(„RScript -e rmarkdown::render(...)“) and fire up a fresh R session for the knitting. This is essentially what RStudio does when you click Knit.
            – AlexR
            Nov 24 at 5:40












          • I'm not a great fan of system calls inside packages. But using the "same idea" and executing it in a PSOCK cluster, it seems to work fine.
            – F. Privé
            Nov 24 at 6:15










          • See minipkg::my_render2().
            – F. Privé
            Nov 24 at 6:19










          • @F.Privé That‘s not much different, maybe a bit more overhead for setting up R in slave mode. Anyway, using a parallel package just hides the creation of the R process - still safer than playing with the envir parameter but I think system or system2 are more efficient and clearer as to why. Plus on linux a (default) FORK cluster will have a copy of globalenv(), so care must be taken to specify the cluster type.
            – AlexR
            Nov 24 at 6:19


















          • Good job at finding the root of the problem. Yet, it feels wrong to add the global environment to the search path for the Rmd. Does this means that the Rmd has access to the current global env? I was thinking about using new.env(parent = parent.env(globalenv())). But this means that the Rmd has access to packages loaded before, right?
            – F. Privé
            Nov 23 at 22:49












          • @F.Privé That might work with already attached packages but new library() calls might modify the search path „above“. Also, source() calls modify globalenv() unless local = TRUE is specified. All in all it‘s probably safer to do system(„RScript -e rmarkdown::render(...)“) and fire up a fresh R session for the knitting. This is essentially what RStudio does when you click Knit.
            – AlexR
            Nov 24 at 5:40












          • I'm not a great fan of system calls inside packages. But using the "same idea" and executing it in a PSOCK cluster, it seems to work fine.
            – F. Privé
            Nov 24 at 6:15










          • See minipkg::my_render2().
            – F. Privé
            Nov 24 at 6:19










          • @F.Privé That‘s not much different, maybe a bit more overhead for setting up R in slave mode. Anyway, using a parallel package just hides the creation of the R process - still safer than playing with the envir parameter but I think system or system2 are more efficient and clearer as to why. Plus on linux a (default) FORK cluster will have a copy of globalenv(), so care must be taken to specify the cluster type.
            – AlexR
            Nov 24 at 6:19
















          Good job at finding the root of the problem. Yet, it feels wrong to add the global environment to the search path for the Rmd. Does this means that the Rmd has access to the current global env? I was thinking about using new.env(parent = parent.env(globalenv())). But this means that the Rmd has access to packages loaded before, right?
          – F. Privé
          Nov 23 at 22:49






          Good job at finding the root of the problem. Yet, it feels wrong to add the global environment to the search path for the Rmd. Does this means that the Rmd has access to the current global env? I was thinking about using new.env(parent = parent.env(globalenv())). But this means that the Rmd has access to packages loaded before, right?
          – F. Privé
          Nov 23 at 22:49














          @F.Privé That might work with already attached packages but new library() calls might modify the search path „above“. Also, source() calls modify globalenv() unless local = TRUE is specified. All in all it‘s probably safer to do system(„RScript -e rmarkdown::render(...)“) and fire up a fresh R session for the knitting. This is essentially what RStudio does when you click Knit.
          – AlexR
          Nov 24 at 5:40






          @F.Privé That might work with already attached packages but new library() calls might modify the search path „above“. Also, source() calls modify globalenv() unless local = TRUE is specified. All in all it‘s probably safer to do system(„RScript -e rmarkdown::render(...)“) and fire up a fresh R session for the knitting. This is essentially what RStudio does when you click Knit.
          – AlexR
          Nov 24 at 5:40














          I'm not a great fan of system calls inside packages. But using the "same idea" and executing it in a PSOCK cluster, it seems to work fine.
          – F. Privé
          Nov 24 at 6:15




          I'm not a great fan of system calls inside packages. But using the "same idea" and executing it in a PSOCK cluster, it seems to work fine.
          – F. Privé
          Nov 24 at 6:15












          See minipkg::my_render2().
          – F. Privé
          Nov 24 at 6:19




          See minipkg::my_render2().
          – F. Privé
          Nov 24 at 6:19












          @F.Privé That‘s not much different, maybe a bit more overhead for setting up R in slave mode. Anyway, using a parallel package just hides the creation of the R process - still safer than playing with the envir parameter but I think system or system2 are more efficient and clearer as to why. Plus on linux a (default) FORK cluster will have a copy of globalenv(), so care must be taken to specify the cluster type.
          – AlexR
          Nov 24 at 6:19




          @F.Privé That‘s not much different, maybe a bit more overhead for setting up R in slave mode. Anyway, using a parallel package just hides the creation of the R process - still safer than playing with the envir parameter but I think system or system2 are more efficient and clearer as to why. Plus on linux a (default) FORK cluster will have a copy of globalenv(), so care must be taken to specify the cluster type.
          – AlexR
          Nov 24 at 6:19


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.





          Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


          Please pay close attention to the following guidance:


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53409789%2frmarkdownrender-problem-when-called-from-a-package%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          404 Error Contact Form 7 ajax form submitting

          How to know if a Active Directory user can login interactively

          TypeError: fit_transform() missing 1 required positional argument: 'X'