Surprisingly difficult Infinite scroll problem

[enoc2222]

Hi All,
I recently found crawl4ai while trying to do some simple scraping on a classified site(https://classifieds.ksl.com).
I am new to scraping and figured this would be a good site to learn/practice some basics on scraping data.
After trying selenium and Requests/Beautifulsoup approach with no luck (apparently they've implemented some decent scraping/bot blockers), I saw a suggestion on reddit to try crawl4ai.
I was able to get a basic script up and kind of running pretty quick and got some initial data but I am hitting a wall trying to get the complete data set from the search.
The classified search result page loads/function like a classic infinite scroll, adding more listings as you scroll down.
With my script, I can usually get the first 11-20 items, sometimes up to 34 (once got 80), but can't ever get the full search results(which as of writing is 254 for the url in the code below).
I've tried virtual scroll but could only ever get the first 11 using the selector ".grid". I've also tried scroll_delay with a range of 0.5 to 10 seconds with no noticeable difference.

I've been banging my head on this till the wee hours and figured wiser minds may have better insight if this is a simple fix or a deeper dive.

Here's my current best working code:

import json
import asyncio
from crawl4ai import AsyncWebCrawler, CrawlerRunConfig, VirtualScrollConfig
from crawl4ai import JsonXPathExtractionStrategy
from crawl4ai.async_configs import CacheMode

async def extract_crypto_prices_xpath():

	schema = {
		"name": "Crypto Prices via XPath",
		#"baseSelector": "//a[contains(@role, 'listitem')]",#Alternate selector
		"baseSelector": "//a[contains(@class, 'grid-flow-col')]",
		"fields": [
			{
				"name": "title",
				"selector": ".//div[contains(@class, 'line-clamp-2')]",
				"type": "text"
			}
		]
	}
	config = CrawlerRunConfig(
		extraction_strategy=JsonXPathExtractionStrategy(schema, verbose=True),
		scan_full_page=True,
	)

	raw_url = "https://classifieds.ksl.com/v2/search/keyword/bin"

	async with AsyncWebCrawler(verbose=False) as crawler:
		result = await crawler.arun(
			url=raw_url,
			config=config,
			magic = True,
		)

		if not result.success:
			print("Crawl failed:", result.error_message)
			return

		data = json.loads(result.extracted_content)

		print(f"Extracted {len(data)} rows")
		for d in data:
			print(f'Title: {d}')

asyncio.run(extract_crypto_prices_xpath())

Let me know if you have any thoughts/suggestions
Thanks!