python - Pivoting a pandas dataframe with duplicate index values -


i have data frame has rows each user joining site , making purchase.

+---+-----+--------------------+---------+--------+-----+ |   | uid |        msg         |  _time  | gender | age | +---+-----+--------------------+---------+--------+-----+ | 0 |   1 | confirmed_settings | 1/29/15 | m      |  37 | | 1 |   1 | sale               | 4/13/15 | m      |  37 | | 2 |   3 | confirmed_settings | 4/19/15 | m      |  35 | | 3 |   4 | confirmed_settings | 2/21/15 | m      |  21 | | 4 |   5 | confirmed_settings | 3/28/15 | m      |  18 | | 5 |   4 | sale               | 3/15/15 | m      |  21 | +---+-----+--------------------+---------+--------+-----+

i change dataframe each row unique uid , there columns called sale , confirmed_settings have timestamp of action. note not every user has sale, every user has confirmed_settings. below:

+---+-----+--------------------+---------+---------+--------+-----+ |   | uid | confirmed_settings |  sale   |  _time  | gender | age | +---+-----+--------------------+---------+---------+--------+-----+ | 0 |   1 | 1/29/15            | 4/13/15 | 1/29/15 | m      |  37 | | 1 |   3 | 4/19/15            | null    | 4/19/15 | m      |  35 | | 2 |   4 | 2/21/15            | 3/15/15 | 2/21/15 | m      |  21 | | 3 |   5 | 3/28/15            | null    | 3/28/15 | m      |  18 | +---+-----+--------------------+---------+---------+--------+-----+

to this, trying:

df1 = df.pivot(index='uid', columns='msg', values='_time').reset_index() df1 = df1.merge(df[['uid', 'gender', 'age']].drop_duplicates(), on='uid')

but error: valueerror: index contains duplicate entries, cannot reshape

how can pivot df duplicate index values transform dataframe?

edit: df1 = df.pivot_table(index='uid', columns='msg', values='_time').reset_index()

gives error dataerror: no numeric types aggregate im not sure right path go on.

x data frame have input :

    uid               msg   _time   gender  age 0   1   confirmed_settings  1/29/15 m       37 1   1   sale                4/13/15 m       37 2   3   confirmed_settings  4/19/15 m       35 3   4   confirmed_settings  2/21/15 m       21 4   5   confirmed_settings  3/28/15 m       18 5   4   sale                3/15/15 m       21  y = x.pivot(index='uid', columns='msg', values='_time') x.join(y).drop('msg', axis=1)

gives you:

    uid _time   gender  age     confirmed_settings  sale 0   1   1/29/15     m   37                    nan   nan 1   1   4/13/15     m   37                1/29/15   4/13/15 2   3   4/19/15     m   35                    nan   nan 3   4   2/21/15     m   21                4/19/15   nan 4   5   3/28/15     m   18                2/21/15   3/15/15 5   4   3/15/15     m   21                3/28/15   nan

Comments

Post a Comment

Popular posts from this blog

java - Spring Data JPA: Why findOne(id) executing delete query internally? -

i have 1 application using @manytomany annotation link 2 different entities 1 parent entity. not sure if have implemented properly. when execute .findone() on parent entity's repository find it, @ first fetch data along executing delete queries internally delete record mapping table (a table jpa creates after adding @manytomany annotation.). following generated sql queries , code snippet: java class @entity @table(name = "inventory") public class inventory { @id long id; // many other properties @manytomany(fetch=fetchtype.lazy) private list<stream> streams; @manytomany(fetch=fetchtype.lazy) private list<subject> subjects; } there no other annotation i've put in stream , subject entity class link inventory class. when run app, create following tables: inventory stream subject inventory_stream inventory_subject (inventory_stream , inventory_subject, 2 tables created automatically, don't k...
Read more

python - Mongodb How to add addtional information when aggregating? -

i beginner , wrote line pipeline works, wanna add other information output, screen name , or number of tweets.i tried add under $group gave me syntax error everytime here pipeline: def make_pipeline(): # complete aggregation pipeline pipeline = [ { '$match': { "user.statuses_count": {"$gt":99 }, "user.time_zone": "brasilia" } }, { "$group": { "_id": "$user.id", "followers": { "$max": "$user.followers_count" } } }, { "$sort": { "followers": -1 } }, { "$limit" : 1 } ]; i using on example : { "_id" : objectid("5304e2e3cc9e684aa98bef97"), "text" : "first week of school on :p", "in_reply_to_status_id" : null, "retweet_count" : null, ...
Read more

java - Incorrect order of records in M-M relationship in hibernate -

Image
i have 4 records in bridge table in mysql database. using @manytomany in hibernate, accurate order of employees incorrect incorrect of projects. following required material figure out doing mistake. this source. @entity @table(name="project") public class projectbean { @id @generatedvalue @column(name="project_id") private int projectid; @column(name="title") private string projecttitle; @manytomany(mappedby="projects") private collection<employeebean> employees; public projectbean() { employees = new arraylist<employeebean>(); } //getters & setters } @entity @table(name="employee") public class employeebean { @id @generatedvalue @column(name="employee_id") private int employee_id; @column(name="first_name") private string ...
Read more