Convert time object to datetime format in python pandas












2















I have a dataset of column name DateTime having dtype object.



df['DateTime'] = pd.to_datetime(df['DateTime'])


I have used the above code to convert to datetime format then did a split in the column to have Date and Time separately



df['date'] = df['DateTime'].dt.date
df['time'] = df['DateTime'].dt.time


but after the split the format changes to object type and while converting it to datetime it showing error for the time column name as: TypeError: is not convertible to datetime



How to convert it to datetime format the time column










share|improve this question

























  • What do you mean "proper time format"...? Please show the code you're using that produces TypeError: is not convertible to datetime

    – Jon Clements
    Nov 25 '18 at 18:02











  • The same code that I have showed to convert the DateTime i.e df['time'] = pd.to_datetime(df.['time'])

    – Nadeem Haque
    Nov 25 '18 at 18:11













  • @NadeemHaque - converting to string is necessary like df['time'] = pd.to_datetime(df.['time'].astype(str)) but then is added some dates, because datetimes with no dates not exist.

    – jezrael
    Nov 25 '18 at 18:29






  • 1





    @jezrael I'm new to python so got confused.. thank you for the help

    – Nadeem Haque
    Nov 25 '18 at 18:59
















2















I have a dataset of column name DateTime having dtype object.



df['DateTime'] = pd.to_datetime(df['DateTime'])


I have used the above code to convert to datetime format then did a split in the column to have Date and Time separately



df['date'] = df['DateTime'].dt.date
df['time'] = df['DateTime'].dt.time


but after the split the format changes to object type and while converting it to datetime it showing error for the time column name as: TypeError: is not convertible to datetime



How to convert it to datetime format the time column










share|improve this question

























  • What do you mean "proper time format"...? Please show the code you're using that produces TypeError: is not convertible to datetime

    – Jon Clements
    Nov 25 '18 at 18:02











  • The same code that I have showed to convert the DateTime i.e df['time'] = pd.to_datetime(df.['time'])

    – Nadeem Haque
    Nov 25 '18 at 18:11













  • @NadeemHaque - converting to string is necessary like df['time'] = pd.to_datetime(df.['time'].astype(str)) but then is added some dates, because datetimes with no dates not exist.

    – jezrael
    Nov 25 '18 at 18:29






  • 1





    @jezrael I'm new to python so got confused.. thank you for the help

    – Nadeem Haque
    Nov 25 '18 at 18:59














2












2








2








I have a dataset of column name DateTime having dtype object.



df['DateTime'] = pd.to_datetime(df['DateTime'])


I have used the above code to convert to datetime format then did a split in the column to have Date and Time separately



df['date'] = df['DateTime'].dt.date
df['time'] = df['DateTime'].dt.time


but after the split the format changes to object type and while converting it to datetime it showing error for the time column name as: TypeError: is not convertible to datetime



How to convert it to datetime format the time column










share|improve this question
















I have a dataset of column name DateTime having dtype object.



df['DateTime'] = pd.to_datetime(df['DateTime'])


I have used the above code to convert to datetime format then did a split in the column to have Date and Time separately



df['date'] = df['DateTime'].dt.date
df['time'] = df['DateTime'].dt.time


but after the split the format changes to object type and while converting it to datetime it showing error for the time column name as: TypeError: is not convertible to datetime



How to convert it to datetime format the time column







python python-3.x pandas datetime dataframe






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 25 '18 at 18:30







Nadeem Haque

















asked Nov 25 '18 at 17:59









Nadeem HaqueNadeem Haque

194




194













  • What do you mean "proper time format"...? Please show the code you're using that produces TypeError: is not convertible to datetime

    – Jon Clements
    Nov 25 '18 at 18:02











  • The same code that I have showed to convert the DateTime i.e df['time'] = pd.to_datetime(df.['time'])

    – Nadeem Haque
    Nov 25 '18 at 18:11













  • @NadeemHaque - converting to string is necessary like df['time'] = pd.to_datetime(df.['time'].astype(str)) but then is added some dates, because datetimes with no dates not exist.

    – jezrael
    Nov 25 '18 at 18:29






  • 1





    @jezrael I'm new to python so got confused.. thank you for the help

    – Nadeem Haque
    Nov 25 '18 at 18:59



















  • What do you mean "proper time format"...? Please show the code you're using that produces TypeError: is not convertible to datetime

    – Jon Clements
    Nov 25 '18 at 18:02











  • The same code that I have showed to convert the DateTime i.e df['time'] = pd.to_datetime(df.['time'])

    – Nadeem Haque
    Nov 25 '18 at 18:11













  • @NadeemHaque - converting to string is necessary like df['time'] = pd.to_datetime(df.['time'].astype(str)) but then is added some dates, because datetimes with no dates not exist.

    – jezrael
    Nov 25 '18 at 18:29






  • 1





    @jezrael I'm new to python so got confused.. thank you for the help

    – Nadeem Haque
    Nov 25 '18 at 18:59

















What do you mean "proper time format"...? Please show the code you're using that produces TypeError: is not convertible to datetime

– Jon Clements
Nov 25 '18 at 18:02





What do you mean "proper time format"...? Please show the code you're using that produces TypeError: is not convertible to datetime

– Jon Clements
Nov 25 '18 at 18:02













The same code that I have showed to convert the DateTime i.e df['time'] = pd.to_datetime(df.['time'])

– Nadeem Haque
Nov 25 '18 at 18:11







The same code that I have showed to convert the DateTime i.e df['time'] = pd.to_datetime(df.['time'])

– Nadeem Haque
Nov 25 '18 at 18:11















@NadeemHaque - converting to string is necessary like df['time'] = pd.to_datetime(df.['time'].astype(str)) but then is added some dates, because datetimes with no dates not exist.

– jezrael
Nov 25 '18 at 18:29





@NadeemHaque - converting to string is necessary like df['time'] = pd.to_datetime(df.['time'].astype(str)) but then is added some dates, because datetimes with no dates not exist.

– jezrael
Nov 25 '18 at 18:29




1




1





@jezrael I'm new to python so got confused.. thank you for the help

– Nadeem Haque
Nov 25 '18 at 18:59





@jezrael I'm new to python so got confused.. thank you for the help

– Nadeem Haque
Nov 25 '18 at 18:59












2 Answers
2






active

oldest

votes


















0














You can use combine in list comprehension with zip:



df = pd.DataFrame({'DateTime': ['2011-01-01 12:48:20', '2014-01-01 12:30:45']})
df['DateTime'] = pd.to_datetime(df['DateTime'])

df['date'] = df['DateTime'].dt.date
df['time'] = df['DateTime'].dt.time

import datetime
df['new'] = [datetime.datetime.combine(a, b) for a, b in zip(df['date'], df['time'])]
print (df)

DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45


Or convert to strings, join together and convert again:



df['new'] = pd.to_datetime(df['date'].astype(str) + ' ' +df['time'].astype(str))
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45


But if use floor for remove times with converting times to timedeltas then use + only:



df['date'] = df['DateTime'].dt.floor('d')
df['time'] = pd.to_timedelta(df['DateTime'].dt.strftime('%H:%M:%S'))

df['new'] = df['date'] + df['time']
print (df)

DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45





share|improve this answer































    0















    How to convert it back to datetime format the time column




    There appears to be a misunderstanding. Pandas datetime series must include date and time components. This is non-negotiable. You can simply use pd.to_datetime without specifying a date and use the default 1900-01-01 date:



    # date from jezrael

    print(pd.to_datetime(df['time'], format='%H:%M:%S'))

    0 1900-01-01 12:48:20
    1 1900-01-01 12:30:45
    Name: time, dtype: datetime64[ns]


    Or use another date component, for example today's date:



    today = pd.Timestamp('today').strftime('%Y-%m-%d')
    print(pd.to_datetime(today + ' ' + df['time'].astype(str)))

    0 2018-11-25 12:48:20
    1 2018-11-25 12:30:45
    Name: time, dtype: datetime64[ns]


    Or recombine from your date and time series:



    print(pd.to_datetime(df['date'].astype(str) + ' ' + df['time'].astype(str)))

    0 2011-01-01 12:48:20
    1 2014-01-01 12:30:45
    dtype: datetime64[ns]





    share|improve this answer
























    • thank you for the help

      – Nadeem Haque
      Nov 25 '18 at 19:01











    • @NadeemHaque, I'm glad my answer helped you. Please consider marking it as correct so it can help other people checking this question also.

      – jpp
      Nov 25 '18 at 20:18













    Your Answer






    StackExchange.ifUsing("editor", function () {
    StackExchange.using("externalEditor", function () {
    StackExchange.using("snippets", function () {
    StackExchange.snippets.init();
    });
    });
    }, "code-snippets");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "1"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53470304%2fconvert-time-object-to-datetime-format-in-python-pandas%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0














    You can use combine in list comprehension with zip:



    df = pd.DataFrame({'DateTime': ['2011-01-01 12:48:20', '2014-01-01 12:30:45']})
    df['DateTime'] = pd.to_datetime(df['DateTime'])

    df['date'] = df['DateTime'].dt.date
    df['time'] = df['DateTime'].dt.time

    import datetime
    df['new'] = [datetime.datetime.combine(a, b) for a, b in zip(df['date'], df['time'])]
    print (df)

    DateTime date time new
    0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
    1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45


    Or convert to strings, join together and convert again:



    df['new'] = pd.to_datetime(df['date'].astype(str) + ' ' +df['time'].astype(str))
    print (df)
    DateTime date time new
    0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
    1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45


    But if use floor for remove times with converting times to timedeltas then use + only:



    df['date'] = df['DateTime'].dt.floor('d')
    df['time'] = pd.to_timedelta(df['DateTime'].dt.strftime('%H:%M:%S'))

    df['new'] = df['date'] + df['time']
    print (df)

    DateTime date time new
    0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
    1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45





    share|improve this answer




























      0














      You can use combine in list comprehension with zip:



      df = pd.DataFrame({'DateTime': ['2011-01-01 12:48:20', '2014-01-01 12:30:45']})
      df['DateTime'] = pd.to_datetime(df['DateTime'])

      df['date'] = df['DateTime'].dt.date
      df['time'] = df['DateTime'].dt.time

      import datetime
      df['new'] = [datetime.datetime.combine(a, b) for a, b in zip(df['date'], df['time'])]
      print (df)

      DateTime date time new
      0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
      1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45


      Or convert to strings, join together and convert again:



      df['new'] = pd.to_datetime(df['date'].astype(str) + ' ' +df['time'].astype(str))
      print (df)
      DateTime date time new
      0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
      1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45


      But if use floor for remove times with converting times to timedeltas then use + only:



      df['date'] = df['DateTime'].dt.floor('d')
      df['time'] = pd.to_timedelta(df['DateTime'].dt.strftime('%H:%M:%S'))

      df['new'] = df['date'] + df['time']
      print (df)

      DateTime date time new
      0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
      1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45





      share|improve this answer


























        0












        0








        0







        You can use combine in list comprehension with zip:



        df = pd.DataFrame({'DateTime': ['2011-01-01 12:48:20', '2014-01-01 12:30:45']})
        df['DateTime'] = pd.to_datetime(df['DateTime'])

        df['date'] = df['DateTime'].dt.date
        df['time'] = df['DateTime'].dt.time

        import datetime
        df['new'] = [datetime.datetime.combine(a, b) for a, b in zip(df['date'], df['time'])]
        print (df)

        DateTime date time new
        0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
        1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45


        Or convert to strings, join together and convert again:



        df['new'] = pd.to_datetime(df['date'].astype(str) + ' ' +df['time'].astype(str))
        print (df)
        DateTime date time new
        0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
        1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45


        But if use floor for remove times with converting times to timedeltas then use + only:



        df['date'] = df['DateTime'].dt.floor('d')
        df['time'] = pd.to_timedelta(df['DateTime'].dt.strftime('%H:%M:%S'))

        df['new'] = df['date'] + df['time']
        print (df)

        DateTime date time new
        0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
        1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45





        share|improve this answer













        You can use combine in list comprehension with zip:



        df = pd.DataFrame({'DateTime': ['2011-01-01 12:48:20', '2014-01-01 12:30:45']})
        df['DateTime'] = pd.to_datetime(df['DateTime'])

        df['date'] = df['DateTime'].dt.date
        df['time'] = df['DateTime'].dt.time

        import datetime
        df['new'] = [datetime.datetime.combine(a, b) for a, b in zip(df['date'], df['time'])]
        print (df)

        DateTime date time new
        0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
        1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45


        Or convert to strings, join together and convert again:



        df['new'] = pd.to_datetime(df['date'].astype(str) + ' ' +df['time'].astype(str))
        print (df)
        DateTime date time new
        0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
        1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45


        But if use floor for remove times with converting times to timedeltas then use + only:



        df['date'] = df['DateTime'].dt.floor('d')
        df['time'] = pd.to_timedelta(df['DateTime'].dt.strftime('%H:%M:%S'))

        df['new'] = df['date'] + df['time']
        print (df)

        DateTime date time new
        0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
        1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45






        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Nov 25 '18 at 18:10









        jezraeljezrael

        342k25297369




        342k25297369

























            0















            How to convert it back to datetime format the time column




            There appears to be a misunderstanding. Pandas datetime series must include date and time components. This is non-negotiable. You can simply use pd.to_datetime without specifying a date and use the default 1900-01-01 date:



            # date from jezrael

            print(pd.to_datetime(df['time'], format='%H:%M:%S'))

            0 1900-01-01 12:48:20
            1 1900-01-01 12:30:45
            Name: time, dtype: datetime64[ns]


            Or use another date component, for example today's date:



            today = pd.Timestamp('today').strftime('%Y-%m-%d')
            print(pd.to_datetime(today + ' ' + df['time'].astype(str)))

            0 2018-11-25 12:48:20
            1 2018-11-25 12:30:45
            Name: time, dtype: datetime64[ns]


            Or recombine from your date and time series:



            print(pd.to_datetime(df['date'].astype(str) + ' ' + df['time'].astype(str)))

            0 2011-01-01 12:48:20
            1 2014-01-01 12:30:45
            dtype: datetime64[ns]





            share|improve this answer
























            • thank you for the help

              – Nadeem Haque
              Nov 25 '18 at 19:01











            • @NadeemHaque, I'm glad my answer helped you. Please consider marking it as correct so it can help other people checking this question also.

              – jpp
              Nov 25 '18 at 20:18


















            0















            How to convert it back to datetime format the time column




            There appears to be a misunderstanding. Pandas datetime series must include date and time components. This is non-negotiable. You can simply use pd.to_datetime without specifying a date and use the default 1900-01-01 date:



            # date from jezrael

            print(pd.to_datetime(df['time'], format='%H:%M:%S'))

            0 1900-01-01 12:48:20
            1 1900-01-01 12:30:45
            Name: time, dtype: datetime64[ns]


            Or use another date component, for example today's date:



            today = pd.Timestamp('today').strftime('%Y-%m-%d')
            print(pd.to_datetime(today + ' ' + df['time'].astype(str)))

            0 2018-11-25 12:48:20
            1 2018-11-25 12:30:45
            Name: time, dtype: datetime64[ns]


            Or recombine from your date and time series:



            print(pd.to_datetime(df['date'].astype(str) + ' ' + df['time'].astype(str)))

            0 2011-01-01 12:48:20
            1 2014-01-01 12:30:45
            dtype: datetime64[ns]





            share|improve this answer
























            • thank you for the help

              – Nadeem Haque
              Nov 25 '18 at 19:01











            • @NadeemHaque, I'm glad my answer helped you. Please consider marking it as correct so it can help other people checking this question also.

              – jpp
              Nov 25 '18 at 20:18
















            0












            0








            0








            How to convert it back to datetime format the time column




            There appears to be a misunderstanding. Pandas datetime series must include date and time components. This is non-negotiable. You can simply use pd.to_datetime without specifying a date and use the default 1900-01-01 date:



            # date from jezrael

            print(pd.to_datetime(df['time'], format='%H:%M:%S'))

            0 1900-01-01 12:48:20
            1 1900-01-01 12:30:45
            Name: time, dtype: datetime64[ns]


            Or use another date component, for example today's date:



            today = pd.Timestamp('today').strftime('%Y-%m-%d')
            print(pd.to_datetime(today + ' ' + df['time'].astype(str)))

            0 2018-11-25 12:48:20
            1 2018-11-25 12:30:45
            Name: time, dtype: datetime64[ns]


            Or recombine from your date and time series:



            print(pd.to_datetime(df['date'].astype(str) + ' ' + df['time'].astype(str)))

            0 2011-01-01 12:48:20
            1 2014-01-01 12:30:45
            dtype: datetime64[ns]





            share|improve this answer














            How to convert it back to datetime format the time column




            There appears to be a misunderstanding. Pandas datetime series must include date and time components. This is non-negotiable. You can simply use pd.to_datetime without specifying a date and use the default 1900-01-01 date:



            # date from jezrael

            print(pd.to_datetime(df['time'], format='%H:%M:%S'))

            0 1900-01-01 12:48:20
            1 1900-01-01 12:30:45
            Name: time, dtype: datetime64[ns]


            Or use another date component, for example today's date:



            today = pd.Timestamp('today').strftime('%Y-%m-%d')
            print(pd.to_datetime(today + ' ' + df['time'].astype(str)))

            0 2018-11-25 12:48:20
            1 2018-11-25 12:30:45
            Name: time, dtype: datetime64[ns]


            Or recombine from your date and time series:



            print(pd.to_datetime(df['date'].astype(str) + ' ' + df['time'].astype(str)))

            0 2011-01-01 12:48:20
            1 2014-01-01 12:30:45
            dtype: datetime64[ns]






            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Nov 25 '18 at 18:28









            jppjpp

            101k2163112




            101k2163112













            • thank you for the help

              – Nadeem Haque
              Nov 25 '18 at 19:01











            • @NadeemHaque, I'm glad my answer helped you. Please consider marking it as correct so it can help other people checking this question also.

              – jpp
              Nov 25 '18 at 20:18





















            • thank you for the help

              – Nadeem Haque
              Nov 25 '18 at 19:01











            • @NadeemHaque, I'm glad my answer helped you. Please consider marking it as correct so it can help other people checking this question also.

              – jpp
              Nov 25 '18 at 20:18



















            thank you for the help

            – Nadeem Haque
            Nov 25 '18 at 19:01





            thank you for the help

            – Nadeem Haque
            Nov 25 '18 at 19:01













            @NadeemHaque, I'm glad my answer helped you. Please consider marking it as correct so it can help other people checking this question also.

            – jpp
            Nov 25 '18 at 20:18







            @NadeemHaque, I'm glad my answer helped you. Please consider marking it as correct so it can help other people checking this question also.

            – jpp
            Nov 25 '18 at 20:18




















            draft saved

            draft discarded




















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53470304%2fconvert-time-object-to-datetime-format-in-python-pandas%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            404 Error Contact Form 7 ajax form submitting

            How to know if a Active Directory user can login interactively

            TypeError: fit_transform() missing 1 required positional argument: 'X'