289,232 questions
0 votes
0 answers
34 views
Suitable Pandas installation on 32-bit Python (3.10, 3.11)
I am trying to use the Kiwoom OpenAPI (for making automated stock trading program), which requires a 32-bit Python environment. However, to successfully use Kiwoom Open API I need to set up pandas in ...
0 votes
1 answer
82 views
Why does groupby().apply() produce inconsistent results on identical groups when the DataFrame has overlapping indices?
I noticed that groupby().apply() produces different results for two groups that look identical, except that the overall DataFrame has duplicate index values. Here is a minimal reproducible example: ...
89 votes
8 answers
89k views
How to query if a list-type column contains something?
I have a dataframe, which contains info about movies. It has a column called genre, which contains a list of genres it belongs to. For example: df['genre'] ## returns 0 ['comedy', 'sci-fi'] 1 ...
0 votes
3 answers
60 views
How to modify mulitple columns applying if else to multiple pandas dataframe columns
I have a dataFrame with columns Age, Salary and others, if I used: df['Age'] = df['Age'].apply(lambda x : x+100 if x>30 else 0) Then I can modify the Age column with the if else condition. Also, if ...
5 votes
2 answers
4k views
Pandas.read_sql throw exception from sqlalchemy: AttributeError: 'Connection' object has no attribute 'exec_driver_sql'
#pd.version is '2.0.2' #sqlalchemy.version is '1.3.7' import pandas as pd from sqlalchemy import create_engine conn_str = 'connection_string' engine = create_engine(conn_str) Yesterday everything ...
0 votes
3 answers
9k views
How to read csv without header in pandas
I use Adj = pd.read_csv('xxxxxx.csv', usecols=["Adj Close"]) to read my csv file and result is looks like: Adj Close 0 0.007427 1 0.002013 2 0.008874 my csv file look like: ...
1 vote
1 answer
463 views
How to synchronize or merge dataframe with JSON data based on timestamp
There are many examples out there on how to merge two pandas dataframe but my problem is that I have two kind of data. data1 is a csv data that I read it with pandas and turn it to dataframe and the ...
3 votes
1 answer
110 views
Use pandas merge_asof to achieve inexact left join
I have two pandas series: right_series Index Value 1 0.1 2 0.2 3 0.3 6 0.6 7 0.7 left_series Index Value 1 0.1 5 0.5 10 1.0 I would like to join right_series on left_series by the indices, such that ...
2 votes
4 answers
149 views
How to split dataframe into multiple sub-dataframes based on column value
I got a dataframe df1 which looks like this: Column1 Column2 13 1 12 1 15 0 16 0 15 1 14 1 12 1 11 0 21 1 45 1 44 0 The 1s indicate that a measurement started, I don't know how many 1s will be in one ...
57 votes
4 answers
133k views
Get all keys from GroupBy object in Pandas
I'm looking for a way to get a list of all the keys in a GroupBy object, but I can't seem to find one via the docs nor through Google. There is definitely a way to access the groups through their ...
4 votes
2 answers
3k views
Using constant memory with pandas xlsxwriter
I'm trying to use the below code to write large pandas dataframes to excel worsheets. If I write it directly, the system is running out of RAM. Is this a viable option or are there any alternatives? ...
-1 votes
1 answer
68 views
Using list to find variables from data in rows [closed]
I am trying to find a combination where it will go through the data to find matched variables for any value from the list First_row is found, any value from the list Second_row and any value from the ...
Advice
1 vote
3 replies
126 views
Best way to clean awkward Excel column headers in python/pandas?
I've got four years of daily school attendance data spread across 40+ Excel files (one for each month) and the sheets are set up in a truly annoying fashion, with each date in one merged cell in the ...
-1 votes
1 answer
142 views
Extracting table from UPS website using BeautifulSoup [closed]
I'm trying to extract the US diesel price surcharge by table id using the below code, and it's only reading until <thead> but not <tbody>. What am I doing wrong? url = 'https://www.ups.com/...
2 votes
2 answers
11k views
Getting Error: "ValueError: 2 is not in range" when working with Pandas [closed]
I have got this error when try split my one column to few columns. But it split on just on one or two columns.If you wanna split on 3,4,5 columns it writes: ValueError ...