Reflection error while creating spark session through Livy












0















I am creating a spark session through LivyClient.
The code for creating the session is -



var client: LivyClient = null
try {
client = new LivyClientBuilder()
.setURI(new URI("http://127.0.0.1:4141"))
.build();
val pi = client.submit(new FileStream()).get
} finally {
client.stop(true)
}


While creating a session, I am getting the error -



Exception in thread "main" scala.reflect.internal.FatalError: object Predef does not have a member classOf
enter image description here



I am using the following versions -




  1. Apache Livy - 0.5.0-incubating

  2. Apache Spark - 2.3.0

  3. Scala - 2.11 (2.11.8)


I have checked the scala-reflect version in all the three and they are same.



I am unable to understand what is causing the below error










share|improve this question























  • Welcome to Stack Overflow. Images of errors are highly discouraged.Please edit your question and include the error as a text.

    – user6910411
    Nov 22 '18 at 15:01













  • you should pass Livy Job to client.submit method

    – morsik
    Nov 22 '18 at 20:28
















0















I am creating a spark session through LivyClient.
The code for creating the session is -



var client: LivyClient = null
try {
client = new LivyClientBuilder()
.setURI(new URI("http://127.0.0.1:4141"))
.build();
val pi = client.submit(new FileStream()).get
} finally {
client.stop(true)
}


While creating a session, I am getting the error -



Exception in thread "main" scala.reflect.internal.FatalError: object Predef does not have a member classOf
enter image description here



I am using the following versions -




  1. Apache Livy - 0.5.0-incubating

  2. Apache Spark - 2.3.0

  3. Scala - 2.11 (2.11.8)


I have checked the scala-reflect version in all the three and they are same.



I am unable to understand what is causing the below error










share|improve this question























  • Welcome to Stack Overflow. Images of errors are highly discouraged.Please edit your question and include the error as a text.

    – user6910411
    Nov 22 '18 at 15:01













  • you should pass Livy Job to client.submit method

    – morsik
    Nov 22 '18 at 20:28














0












0








0








I am creating a spark session through LivyClient.
The code for creating the session is -



var client: LivyClient = null
try {
client = new LivyClientBuilder()
.setURI(new URI("http://127.0.0.1:4141"))
.build();
val pi = client.submit(new FileStream()).get
} finally {
client.stop(true)
}


While creating a session, I am getting the error -



Exception in thread "main" scala.reflect.internal.FatalError: object Predef does not have a member classOf
enter image description here



I am using the following versions -




  1. Apache Livy - 0.5.0-incubating

  2. Apache Spark - 2.3.0

  3. Scala - 2.11 (2.11.8)


I have checked the scala-reflect version in all the three and they are same.



I am unable to understand what is causing the below error










share|improve this question














I am creating a spark session through LivyClient.
The code for creating the session is -



var client: LivyClient = null
try {
client = new LivyClientBuilder()
.setURI(new URI("http://127.0.0.1:4141"))
.build();
val pi = client.submit(new FileStream()).get
} finally {
client.stop(true)
}


While creating a session, I am getting the error -



Exception in thread "main" scala.reflect.internal.FatalError: object Predef does not have a member classOf
enter image description here



I am using the following versions -




  1. Apache Livy - 0.5.0-incubating

  2. Apache Spark - 2.3.0

  3. Scala - 2.11 (2.11.8)


I have checked the scala-reflect version in all the three and they are same.



I am unable to understand what is causing the below error







scala apache-spark livy






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 22 '18 at 14:49









Naina JainNaina Jain

1




1













  • Welcome to Stack Overflow. Images of errors are highly discouraged.Please edit your question and include the error as a text.

    – user6910411
    Nov 22 '18 at 15:01













  • you should pass Livy Job to client.submit method

    – morsik
    Nov 22 '18 at 20:28



















  • Welcome to Stack Overflow. Images of errors are highly discouraged.Please edit your question and include the error as a text.

    – user6910411
    Nov 22 '18 at 15:01













  • you should pass Livy Job to client.submit method

    – morsik
    Nov 22 '18 at 20:28

















Welcome to Stack Overflow. Images of errors are highly discouraged.Please edit your question and include the error as a text.

– user6910411
Nov 22 '18 at 15:01







Welcome to Stack Overflow. Images of errors are highly discouraged.Please edit your question and include the error as a text.

– user6910411
Nov 22 '18 at 15:01















you should pass Livy Job to client.submit method

– morsik
Nov 22 '18 at 20:28





you should pass Livy Job to client.submit method

– morsik
Nov 22 '18 at 20:28












1 Answer
1






active

oldest

votes


















1














What version of Java are you using? It appears that changes in Java 9 regarding dropping of deprecated features stops Spark with errors referring to reflection. The home page says that the Java must be at least version 8.






share|improve this answer























    Your Answer






    StackExchange.ifUsing("editor", function () {
    StackExchange.using("externalEditor", function () {
    StackExchange.using("snippets", function () {
    StackExchange.snippets.init();
    });
    });
    }, "code-snippets");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "1"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53433459%2freflection-error-while-creating-spark-session-through-livy%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    1














    What version of Java are you using? It appears that changes in Java 9 regarding dropping of deprecated features stops Spark with errors referring to reflection. The home page says that the Java must be at least version 8.






    share|improve this answer




























      1














      What version of Java are you using? It appears that changes in Java 9 regarding dropping of deprecated features stops Spark with errors referring to reflection. The home page says that the Java must be at least version 8.






      share|improve this answer


























        1












        1








        1







        What version of Java are you using? It appears that changes in Java 9 regarding dropping of deprecated features stops Spark with errors referring to reflection. The home page says that the Java must be at least version 8.






        share|improve this answer













        What version of Java are you using? It appears that changes in Java 9 regarding dropping of deprecated features stops Spark with errors referring to reflection. The home page says that the Java must be at least version 8.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Dec 11 '18 at 15:48









        Bradley RossBradley Ross

        43028




        43028






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53433459%2freflection-error-while-creating-spark-session-through-livy%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            404 Error Contact Form 7 ajax form submitting

            How to know if a Active Directory user can login interactively

            TypeError: fit_transform() missing 1 required positional argument: 'X'