Pandas question

  • Thread starter Albert-Jan Roskam
  • Start date
A

Albert-Jan Roskam

Hi,

I am new to Pandas. I am trying to remove the lower and upper 15 percentiles of interest rates within a day. The index column is the date. Below is some code, but how do I apply the trim function day-by-day? I tried using grouped() in conjunction with apply(), but that turned out to be an efficient way to slow my computer down and choke it. Any thoughts?


import pandas as pd

records = pd.read_table("blah.csv", sep=";", parse_dates=[2], index_col=2, low_memory=False)


def trim(df, colname, boundaries=(0.15, 0.85)):
    lo = df[colname] >= df[colname].quantile(boundaries[0])
    hi = df[colname] <= df[colname].quantile(boundaries[1])
    return df[lo & hi]

trimmed = trim(records, 'pct_12m') # this trims across all data, not day-by-day, which I want.

Oh, and is something like the following possible insteadof df[lo & hi]?

df[lo <= df[colname] <= hi]  # looks nice, but gives ValueError

 
Thanks in advance!


Regards,

Albert-Jan




~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

All right, but apart from the sanitation, the medicine,education, wine, public order, irrigation, roads, a

fresh water system, and public health, what have the Romans ever done for us?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads


Members online

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,901
Latest member
Noble71S45

Latest Threads

Top