FastFrame anti-pattern report

sales_analysis_slow.py · deterministic rewrite and subprocess verification

PandasNumPySafe rewritesBenchmarked

🐼 Install from PyPI

python -m pip install fastframe-pandas pandas numpy
 fastframe scan my_script.py

🐳 Run with Docker

docker build -t fastframe:latest .
  docker run --rm --network none -v "$PWD":/workspace -w /workspace fastframe:latest scan my_script.py

6patterns detected

0.2976soriginal median

0.0221srewritten median

13.48×speedup

+1,891 bytes (+0.0%)peak memory change

PASSresult verification

AP001 · row-wise DataFrame.apply arithmetic

PASS

Line 8, column 9 · Estimated saving: n/a bytes/value

VerifyEquivalent outputs

Speedup155.80×

Memory+17,050,831 bytes (+84.0%)

Timing median0.2428s → 0.0016s

Timing min0.2428s → 0.0016s

PASS

Before

rng = np.random.default_rng(1001)
df = pd.DataFrame({"a": rng.integers(1, 100, 100_000), "b": rng.integers(1, 100, 100_000)})
result = df.apply(lambda row: row["a"] + row["b"], axis=1)

After

rng = np.random.default_rng(1001)
df = pd.DataFrame({"a": rng.integers(1, 100, 100_000), "b": rng.integers(1, 100, 100_000)})
result = df["a"] + df["b"]

Diff

	Before			After
f	1	rng = np.random.default_rng(1001)	f	1	rng = np.random.default_rng(1001)
	2	df = pd.DataFrame({"a": rng.integers(1, 100, 100_000), "b": rng.integers(1, 100, 100_000)})		2	df = pd.DataFrame({"a": rng.integers(1, 100, 100_000), "b": rng.integers(1, 100, 100_000)})
t	3	result = df.apply(lambda row: row["a"] + row["b"], axis=1)	t	3	result = df["a"] + df["b"]

AP002 · Series.apply arithmetic lambda

PASS

Line 8, column 9 · Estimated saving: n/a bytes/value

VerifyEquivalent outputs

Speedup14.24×

Memory+5,000,640 bytes (+75.1%)

Timing median0.0156s → 0.0011s

Timing min0.0156s → 0.0011s

PASS

Before

rng = np.random.default_rng(1002)
df = pd.DataFrame({"a": rng.integers(1, 100, 100_000)})
result = df["a"].apply(lambda value: value * 2)

After

rng = np.random.default_rng(1002)
df = pd.DataFrame({"a": rng.integers(1, 100, 100_000)})
result = df["a"] * 2

Diff

	Before			After
f	1	rng = np.random.default_rng(1002)	f	1	rng = np.random.default_rng(1002)
	2	df = pd.DataFrame({"a": rng.integers(1, 100, 100_000)})		2	df = pd.DataFrame({"a": rng.integers(1, 100, 100_000)})
t	3	result = df["a"].apply(lambda value: value * 2)	t	3	result = df["a"] * 2

AP003 · iterrows arithmetic append-to-list

PASS

Line 9, column 0 · Estimated saving: n/a bytes/value

VerifyEquivalent outputs

Speedup342.11×

Memory+1,599,495 bytes (+32.9%)

Timing median0.8118s → 0.0024s

Timing min0.8118s → 0.0024s

PASS

Before

df = pd.DataFrame({"a": rng.integers(1, 100, 100_000), "b": rng.integers(1, 100, 100_000)})
result = []
for _, row in df.iterrows():
    result.append(row["a"] + row["b"])

After

df = pd.DataFrame({"a": rng.integers(1, 100, 100_000), "b": rng.integers(1, 100, 100_000)})
result = []
result.extend((df["a"] + df["b"]).tolist())
if len(df):
    row = df.iloc[-1]

Diff

	Before			After
f	1	df = pd.DataFrame({"a": rng.integers(1, 100, 100_000), "b": rng.integers(1, 100, 100_000)})	f	1	df = pd.DataFrame({"a": rng.integers(1, 100, 100_000), "b": rng.integers(1, 100, 100_000)})
	2	result = []		2	result = []
t	3	for _, row in df.iterrows():	t	3	result.extend((df["a"] + df["b"]).tolist())
	4	result.append(row["a"] + row["b"])		4	if len(df):
				5	row = df.iloc[-1]

AP004 · growing DataFrame with concat/append in a loop

FAIL

Line 9, column 0 · Estimated saving: n/a bytes/value

VerifyThe truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

Speedup7.65×

Memory-1,116,041 bytes (-3.6%)

Timing median0.7046s → 0.0921s

Timing min0.7046s → 0.0921s

FAIL

Before

chunks = [pd.DataFrame({"value": rng.integers(1, 100, 500)}) for _ in range(2_000)]
result = pd.DataFrame({"value": []})
for chunk in chunks:
    result = pd.concat([result, chunk], ignore_index=True)

After

chunks = [pd.DataFrame({"value": rng.integers(1, 100, 500)}) for _ in range(2_000)]
result = pd.DataFrame({"value": []})
_fastframe_result_parts = [result]
for chunk in chunks:
    _fastframe_result_parts.append(chunk)

Diff

	Before			After
f	1	chunks = [pd.DataFrame({"value": rng.integers(1, 100, 500)}) for _ in range(2_000)]	f	1	chunks = [pd.DataFrame({"value": rng.integers(1, 100, 500)}) for _ in range(2_000)]
	2	result = pd.DataFrame({"value": []})		2	result = pd.DataFrame({"value": []})
n			n	3	_fastframe_result_parts = [result]
	3	for chunk in chunks:		4	for chunk in chunks:
t	4	result = pd.concat([result, chunk], ignore_index=True)	t	5	_fastframe_result_parts.append(chunk)

AP005 · growing NumPy array with np.append in a loop

PASS

Line 8, column 0 · Estimated saving: n/a bytes/value

VerifyEquivalent outputs

Speedup7.06×

Memory-4,812,636 bytes (-919.3%)

Timing median0.0807s → 0.0114s

Timing min0.0807s → 0.0114s

PASS

Before

values = rng.integers(1, 100, 20_000)
result = np.array([], dtype=np.int64)
for value in values:
    result = np.append(result, value)

After

values = rng.integers(1, 100, 20_000)
result = np.array([], dtype=np.int64)
_fastframe_result_parts = [result]
for value in values:
    _fastframe_result_parts.append(np.atleast_1d(value))

Diff

	Before			After
f	1	values = rng.integers(1, 100, 20_000)	f	1	values = rng.integers(1, 100, 20_000)
	2	result = np.array([], dtype=np.int64)		2	result = np.array([], dtype=np.int64)
n			n	3	_fastframe_result_parts = [result]
	3	for value in values:		4	for value in values:
t	4	result = np.append(result, value)	t	5	_fastframe_result_parts.append(np.atleast_1d(value))

AP006 · downcastable object/int64/float64 column (manual review)

MANUAL REVIEW

Line 32, column 13 · Estimated saving: 4 bytes/value

VerifyNo code rewrite; advisory finding only

Speedupadvisory only

Memoryadvisory only

PASS · manual review

Before

    amounts = np.append(amounts, value)

wide_units = work["units"].astype("int64")

After

    amounts = np.append(amounts, value)

wide_units = work["units"].astype("int64")