How to find the row index of the first occurrence of a match in a cell in Python dataframe (containing date)












1















I have a Python data frame containing a column with Date Time like this
2019-01-02 09:00:00 (which means January 2, 2019 9 AM)



There may be a bunch of rows which have the same date in the Date Time column.



In other words, I can have 2019-01-02 09:00:00 or 2019-01-02 09:15:00 or 2019-01-02 09:30:00 and so on.



Now I need to find the row index of the first occurrence of the date 2019-01-02 in the Python data frame.



I obviously do this using a loop, but am wondering if there is a better way.



With the df['Date Time'].str.contains() method, I can get that all the rows that match a given date, but I need the index.



The generic question is that how do we find the index of a first occurrence of a match in a cell in Python data frame that matches a given string pattern.



The more specific question is that how do we find the index of a first occurrence of a match in a cell in Python data frame that matches a given date in a cell that contains date Time assuming that the Python data frame is sorted in chronologically ascending order of date Time , i.e.
2019-01-02 09:00:00 occurs at an index earlier than 2019-01-02 09:15:00 followed by 2019-01-03 09:00:00 and so on.



Thank you for any inputs










share|improve this question


















  • 1





    can you not just call the index function?: df[df['Date Time'].dt.date == pd.Timestamp('2019-01-02').date()].head(1).index

    – Chris
    Jan 18 at 14:36













  • Better , you can try creating a DataFrame sample at least and try with that doesn't matter if it works or not in order to show us the data so, you can get appropriate answer for your requirement , text details doesn't create a good understanding.

    – pygo
    Jan 18 at 14:40


















1















I have a Python data frame containing a column with Date Time like this
2019-01-02 09:00:00 (which means January 2, 2019 9 AM)



There may be a bunch of rows which have the same date in the Date Time column.



In other words, I can have 2019-01-02 09:00:00 or 2019-01-02 09:15:00 or 2019-01-02 09:30:00 and so on.



Now I need to find the row index of the first occurrence of the date 2019-01-02 in the Python data frame.



I obviously do this using a loop, but am wondering if there is a better way.



With the df['Date Time'].str.contains() method, I can get that all the rows that match a given date, but I need the index.



The generic question is that how do we find the index of a first occurrence of a match in a cell in Python data frame that matches a given string pattern.



The more specific question is that how do we find the index of a first occurrence of a match in a cell in Python data frame that matches a given date in a cell that contains date Time assuming that the Python data frame is sorted in chronologically ascending order of date Time , i.e.
2019-01-02 09:00:00 occurs at an index earlier than 2019-01-02 09:15:00 followed by 2019-01-03 09:00:00 and so on.



Thank you for any inputs










share|improve this question


















  • 1





    can you not just call the index function?: df[df['Date Time'].dt.date == pd.Timestamp('2019-01-02').date()].head(1).index

    – Chris
    Jan 18 at 14:36













  • Better , you can try creating a DataFrame sample at least and try with that doesn't matter if it works or not in order to show us the data so, you can get appropriate answer for your requirement , text details doesn't create a good understanding.

    – pygo
    Jan 18 at 14:40
















1












1








1








I have a Python data frame containing a column with Date Time like this
2019-01-02 09:00:00 (which means January 2, 2019 9 AM)



There may be a bunch of rows which have the same date in the Date Time column.



In other words, I can have 2019-01-02 09:00:00 or 2019-01-02 09:15:00 or 2019-01-02 09:30:00 and so on.



Now I need to find the row index of the first occurrence of the date 2019-01-02 in the Python data frame.



I obviously do this using a loop, but am wondering if there is a better way.



With the df['Date Time'].str.contains() method, I can get that all the rows that match a given date, but I need the index.



The generic question is that how do we find the index of a first occurrence of a match in a cell in Python data frame that matches a given string pattern.



The more specific question is that how do we find the index of a first occurrence of a match in a cell in Python data frame that matches a given date in a cell that contains date Time assuming that the Python data frame is sorted in chronologically ascending order of date Time , i.e.
2019-01-02 09:00:00 occurs at an index earlier than 2019-01-02 09:15:00 followed by 2019-01-03 09:00:00 and so on.



Thank you for any inputs










share|improve this question














I have a Python data frame containing a column with Date Time like this
2019-01-02 09:00:00 (which means January 2, 2019 9 AM)



There may be a bunch of rows which have the same date in the Date Time column.



In other words, I can have 2019-01-02 09:00:00 or 2019-01-02 09:15:00 or 2019-01-02 09:30:00 and so on.



Now I need to find the row index of the first occurrence of the date 2019-01-02 in the Python data frame.



I obviously do this using a loop, but am wondering if there is a better way.



With the df['Date Time'].str.contains() method, I can get that all the rows that match a given date, but I need the index.



The generic question is that how do we find the index of a first occurrence of a match in a cell in Python data frame that matches a given string pattern.



The more specific question is that how do we find the index of a first occurrence of a match in a cell in Python data frame that matches a given date in a cell that contains date Time assuming that the Python data frame is sorted in chronologically ascending order of date Time , i.e.
2019-01-02 09:00:00 occurs at an index earlier than 2019-01-02 09:15:00 followed by 2019-01-03 09:00:00 and so on.



Thank you for any inputs







python pandas date






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Jan 18 at 14:20









RamanaRamana

256




256








  • 1





    can you not just call the index function?: df[df['Date Time'].dt.date == pd.Timestamp('2019-01-02').date()].head(1).index

    – Chris
    Jan 18 at 14:36













  • Better , you can try creating a DataFrame sample at least and try with that doesn't matter if it works or not in order to show us the data so, you can get appropriate answer for your requirement , text details doesn't create a good understanding.

    – pygo
    Jan 18 at 14:40
















  • 1





    can you not just call the index function?: df[df['Date Time'].dt.date == pd.Timestamp('2019-01-02').date()].head(1).index

    – Chris
    Jan 18 at 14:36













  • Better , you can try creating a DataFrame sample at least and try with that doesn't matter if it works or not in order to show us the data so, you can get appropriate answer for your requirement , text details doesn't create a good understanding.

    – pygo
    Jan 18 at 14:40










1




1





can you not just call the index function?: df[df['Date Time'].dt.date == pd.Timestamp('2019-01-02').date()].head(1).index

– Chris
Jan 18 at 14:36







can you not just call the index function?: df[df['Date Time'].dt.date == pd.Timestamp('2019-01-02').date()].head(1).index

– Chris
Jan 18 at 14:36















Better , you can try creating a DataFrame sample at least and try with that doesn't matter if it works or not in order to show us the data so, you can get appropriate answer for your requirement , text details doesn't create a good understanding.

– pygo
Jan 18 at 14:40







Better , you can try creating a DataFrame sample at least and try with that doesn't matter if it works or not in order to show us the data so, you can get appropriate answer for your requirement , text details doesn't create a good understanding.

– pygo
Jan 18 at 14:40














3 Answers
3






active

oldest

votes


















2














You can use next with iter for first index value matched condition for prevent failed if no matched values:



df = pd.DataFrame({'dates':pd.date_range(start='2018-01-01 20:00:00',
end='2018-01-02 02:00:00', freq='H')})
print (df)
dates
0 2018-01-01 20:00:00
1 2018-01-01 21:00:00
2 2018-01-01 22:00:00
3 2018-01-01 23:00:00
4 2018-01-02 00:00:00
5 2018-01-02 01:00:00
6 2018-01-02 02:00:00

date = '2018-01-02'
mask = df['dates'] >= date
idx = next(iter(mask.index[mask]), 'not exist')
print (idx)
4


date = '2018-01-08'
mask = df['dates'] >= date
idx = next(iter(mask.index[mask]), 'not exist')
print (idx)
not exist


If performance is important, see Efficiently return the index of the first value satisfying condition in array.






share|improve this answer































    1














    Yep you can use .loc and a condition to slice the df, and then return the index using .iloc.



    import pandas as pd
    df = pd.DataFrame({'time':pd.date_range(start='2018-01-01 00:00:00',end='2018-12-31 00:00:00', freq='H')}, index=None).reset_index(drop=True)

    # then use conditions and .iloc to get the first instance
    df.loc[df['time']>'2018-10-30 01:00:00'].iloc[[0,]].index[0]

    # if you specify a coarser condition, for instance without time,
    # it will also return the first instance
    df.loc[df['time']>'2018-10-30'].iloc[[0,]].index[0]





    share|improve this answer































      0














      I do not know, if it is optimal, but it works



      (df['Date Time'].dt.strftime('%Y-%m-%d') == '2019-01-02').idxmax()





      share|improve this answer























        Your Answer






        StackExchange.ifUsing("editor", function () {
        StackExchange.using("externalEditor", function () {
        StackExchange.using("snippets", function () {
        StackExchange.snippets.init();
        });
        });
        }, "code-snippets");

        StackExchange.ready(function() {
        var channelOptions = {
        tags: "".split(" "),
        id: "1"
        };
        initTagRenderer("".split(" "), "".split(" "), channelOptions);

        StackExchange.using("externalEditor", function() {
        // Have to fire editor after snippets, if snippets enabled
        if (StackExchange.settings.snippets.snippetsEnabled) {
        StackExchange.using("snippets", function() {
        createEditor();
        });
        }
        else {
        createEditor();
        }
        });

        function createEditor() {
        StackExchange.prepareEditor({
        heartbeatType: 'answer',
        autoActivateHeartbeat: false,
        convertImagesToLinks: true,
        noModals: true,
        showLowRepImageUploadWarning: true,
        reputationToPostImages: 10,
        bindNavPrevention: true,
        postfix: "",
        imageUploader: {
        brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
        contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
        allowUrls: true
        },
        onDemand: true,
        discardSelector: ".discard-answer"
        ,immediatelyShowMarkdownHelp:true
        });


        }
        });














        draft saved

        draft discarded


















        StackExchange.ready(
        function () {
        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54255890%2fhow-to-find-the-row-index-of-the-first-occurrence-of-a-match-in-a-cell-in-python%23new-answer', 'question_page');
        }
        );

        Post as a guest















        Required, but never shown

























        3 Answers
        3






        active

        oldest

        votes








        3 Answers
        3






        active

        oldest

        votes









        active

        oldest

        votes






        active

        oldest

        votes









        2














        You can use next with iter for first index value matched condition for prevent failed if no matched values:



        df = pd.DataFrame({'dates':pd.date_range(start='2018-01-01 20:00:00',
        end='2018-01-02 02:00:00', freq='H')})
        print (df)
        dates
        0 2018-01-01 20:00:00
        1 2018-01-01 21:00:00
        2 2018-01-01 22:00:00
        3 2018-01-01 23:00:00
        4 2018-01-02 00:00:00
        5 2018-01-02 01:00:00
        6 2018-01-02 02:00:00

        date = '2018-01-02'
        mask = df['dates'] >= date
        idx = next(iter(mask.index[mask]), 'not exist')
        print (idx)
        4


        date = '2018-01-08'
        mask = df['dates'] >= date
        idx = next(iter(mask.index[mask]), 'not exist')
        print (idx)
        not exist


        If performance is important, see Efficiently return the index of the first value satisfying condition in array.






        share|improve this answer




























          2














          You can use next with iter for first index value matched condition for prevent failed if no matched values:



          df = pd.DataFrame({'dates':pd.date_range(start='2018-01-01 20:00:00',
          end='2018-01-02 02:00:00', freq='H')})
          print (df)
          dates
          0 2018-01-01 20:00:00
          1 2018-01-01 21:00:00
          2 2018-01-01 22:00:00
          3 2018-01-01 23:00:00
          4 2018-01-02 00:00:00
          5 2018-01-02 01:00:00
          6 2018-01-02 02:00:00

          date = '2018-01-02'
          mask = df['dates'] >= date
          idx = next(iter(mask.index[mask]), 'not exist')
          print (idx)
          4


          date = '2018-01-08'
          mask = df['dates'] >= date
          idx = next(iter(mask.index[mask]), 'not exist')
          print (idx)
          not exist


          If performance is important, see Efficiently return the index of the first value satisfying condition in array.






          share|improve this answer


























            2












            2








            2







            You can use next with iter for first index value matched condition for prevent failed if no matched values:



            df = pd.DataFrame({'dates':pd.date_range(start='2018-01-01 20:00:00',
            end='2018-01-02 02:00:00', freq='H')})
            print (df)
            dates
            0 2018-01-01 20:00:00
            1 2018-01-01 21:00:00
            2 2018-01-01 22:00:00
            3 2018-01-01 23:00:00
            4 2018-01-02 00:00:00
            5 2018-01-02 01:00:00
            6 2018-01-02 02:00:00

            date = '2018-01-02'
            mask = df['dates'] >= date
            idx = next(iter(mask.index[mask]), 'not exist')
            print (idx)
            4


            date = '2018-01-08'
            mask = df['dates'] >= date
            idx = next(iter(mask.index[mask]), 'not exist')
            print (idx)
            not exist


            If performance is important, see Efficiently return the index of the first value satisfying condition in array.






            share|improve this answer













            You can use next with iter for first index value matched condition for prevent failed if no matched values:



            df = pd.DataFrame({'dates':pd.date_range(start='2018-01-01 20:00:00',
            end='2018-01-02 02:00:00', freq='H')})
            print (df)
            dates
            0 2018-01-01 20:00:00
            1 2018-01-01 21:00:00
            2 2018-01-01 22:00:00
            3 2018-01-01 23:00:00
            4 2018-01-02 00:00:00
            5 2018-01-02 01:00:00
            6 2018-01-02 02:00:00

            date = '2018-01-02'
            mask = df['dates'] >= date
            idx = next(iter(mask.index[mask]), 'not exist')
            print (idx)
            4


            date = '2018-01-08'
            mask = df['dates'] >= date
            idx = next(iter(mask.index[mask]), 'not exist')
            print (idx)
            not exist


            If performance is important, see Efficiently return the index of the first value satisfying condition in array.







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Jan 18 at 15:37









            jezraeljezrael

            328k23270348




            328k23270348

























                1














                Yep you can use .loc and a condition to slice the df, and then return the index using .iloc.



                import pandas as pd
                df = pd.DataFrame({'time':pd.date_range(start='2018-01-01 00:00:00',end='2018-12-31 00:00:00', freq='H')}, index=None).reset_index(drop=True)

                # then use conditions and .iloc to get the first instance
                df.loc[df['time']>'2018-10-30 01:00:00'].iloc[[0,]].index[0]

                # if you specify a coarser condition, for instance without time,
                # it will also return the first instance
                df.loc[df['time']>'2018-10-30'].iloc[[0,]].index[0]





                share|improve this answer




























                  1














                  Yep you can use .loc and a condition to slice the df, and then return the index using .iloc.



                  import pandas as pd
                  df = pd.DataFrame({'time':pd.date_range(start='2018-01-01 00:00:00',end='2018-12-31 00:00:00', freq='H')}, index=None).reset_index(drop=True)

                  # then use conditions and .iloc to get the first instance
                  df.loc[df['time']>'2018-10-30 01:00:00'].iloc[[0,]].index[0]

                  # if you specify a coarser condition, for instance without time,
                  # it will also return the first instance
                  df.loc[df['time']>'2018-10-30'].iloc[[0,]].index[0]





                  share|improve this answer


























                    1












                    1








                    1







                    Yep you can use .loc and a condition to slice the df, and then return the index using .iloc.



                    import pandas as pd
                    df = pd.DataFrame({'time':pd.date_range(start='2018-01-01 00:00:00',end='2018-12-31 00:00:00', freq='H')}, index=None).reset_index(drop=True)

                    # then use conditions and .iloc to get the first instance
                    df.loc[df['time']>'2018-10-30 01:00:00'].iloc[[0,]].index[0]

                    # if you specify a coarser condition, for instance without time,
                    # it will also return the first instance
                    df.loc[df['time']>'2018-10-30'].iloc[[0,]].index[0]





                    share|improve this answer













                    Yep you can use .loc and a condition to slice the df, and then return the index using .iloc.



                    import pandas as pd
                    df = pd.DataFrame({'time':pd.date_range(start='2018-01-01 00:00:00',end='2018-12-31 00:00:00', freq='H')}, index=None).reset_index(drop=True)

                    # then use conditions and .iloc to get the first instance
                    df.loc[df['time']>'2018-10-30 01:00:00'].iloc[[0,]].index[0]

                    # if you specify a coarser condition, for instance without time,
                    # it will also return the first instance
                    df.loc[df['time']>'2018-10-30'].iloc[[0,]].index[0]






                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered Jan 18 at 15:20









                    BenPBenP

                    1941114




                    1941114























                        0














                        I do not know, if it is optimal, but it works



                        (df['Date Time'].dt.strftime('%Y-%m-%d') == '2019-01-02').idxmax()





                        share|improve this answer




























                          0














                          I do not know, if it is optimal, but it works



                          (df['Date Time'].dt.strftime('%Y-%m-%d') == '2019-01-02').idxmax()





                          share|improve this answer


























                            0












                            0








                            0







                            I do not know, if it is optimal, but it works



                            (df['Date Time'].dt.strftime('%Y-%m-%d') == '2019-01-02').idxmax()





                            share|improve this answer













                            I do not know, if it is optimal, but it works



                            (df['Date Time'].dt.strftime('%Y-%m-%d') == '2019-01-02').idxmax()






                            share|improve this answer












                            share|improve this answer



                            share|improve this answer










                            answered Jan 18 at 15:12









                            corscors

                            28718




                            28718






























                                draft saved

                                draft discarded




















































                                Thanks for contributing an answer to Stack Overflow!


                                • Please be sure to answer the question. Provide details and share your research!

                                But avoid



                                • Asking for help, clarification, or responding to other answers.

                                • Making statements based on opinion; back them up with references or personal experience.


                                To learn more, see our tips on writing great answers.




                                draft saved


                                draft discarded














                                StackExchange.ready(
                                function () {
                                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54255890%2fhow-to-find-the-row-index-of-the-first-occurrence-of-a-match-in-a-cell-in-python%23new-answer', 'question_page');
                                }
                                );

                                Post as a guest















                                Required, but never shown





















































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown

































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown







                                Popular posts from this blog

                                Liquibase includeAll doesn't find base path

                                How to use setInterval in EJS file?

                                Petrus Granier-Deferre