Credit Karma caching with variables example

python/credit-karma-cache/.env.example

@@ -0,0 +1,6 @@
+# Browserbase Configuration
+BROWSERBASE_PROJECT_ID=your_browserbase_project_id
+BROWSERBASE_API_KEY=your_browserbase_api_key
+
+# Model API Key (for Gemini)
+GOOGLE_API_KEY=your_google_api_key

python/credit-karma-cache/README.md

@@ -0,0 +1,95 @@
+# Stagehand + Browserbase: Credit Karma with Caching & Variables
+
+## AT A GLANCE
+
+- **Goal**: Automate Credit Karma mortgage rate comparisons using caching and parameterized actions
+- **Key Features**:
+  - Cache DOM snapshots for 10x faster subsequent runs
+  - Parameterize form inputs with variables for flexible automation
+  - Get mortgage refinance rates programmatically
+- **Tech Stack**: Stagehand, Browserbase, Python
+- **Docs** → [Stagehand](https://docs.stagehand.dev) | [Browserbase](https://docs.browserbase.com)
+
+## GLOSSARY
+
+- **Caching**: Stagehand stores DOM snapshots in a local cache directory to dramatically speed up repeat automations. On subsequent runs, Stagehand uses cached DOM state instead of re-analyzing pages from scratch. Docs → [Caching Guide](https://docs.stagehand.dev/v3/best-practices/caching)
+- **Variables**: Parameterize your `act()` instructions with `%variableName%` syntax to make automations reusable and dynamic. Docs → [act() with Variables](https://docs.stagehand.dev/v3/basics/act)
+- **act()**: Stagehand's natural language instruction method that performs UI actions (clicking, typing, selecting) without selectors. Docs → [act() Method](https://docs.stagehand.dev/v3/basics/act)
+- **Browserbase**: Cloud browser infrastructure for reliable, scalable web automation with stealth mode and CAPTCHA solving. Docs → [Browserbase Platform](https://docs.browserbase.com)
+
+## QUICKSTART
+
+1. cd python/credit-karma-cache
+2. uv venv venv
+3. source venv/bin/activate # On Windows: venv\Scripts\activate
+4. uv pip install .
+5. cp .env.example .env # Add your Browserbase API key and Project ID to .env
+6. python main.py
+
+## EXPECTED OUTPUT
+
+On the first run, you'll see:
+```
+🏠 Starting Credit Karma mortgage rate automation...
+
+✓ Successfully navigated to Credit Karma mortgage rates page
+✓ Selected Refinance tab
+✓ Selected credit score: Above 760
+✓ Entered ZIP code: 10001
+✓ Entered loan balance: $100000
+✓ Entered home value: $150000
+✓ Entered cash-out amount: $0
+✓ Clicked 'Get my rates' button
+
+✓ Mortgage rates loaded successfully!
+
+┌─────────────────────────────────────┐
+│ Credit Karma Refinance Query        │
+├─────────────────────────────────────┤
+│ Credit Score: Above 760             │
+│ ZIP Code: 10001                     │
+│ Loan Balance: $100000               │
+│ Home Value: $150000                 │
+│ Cash Out: $0                        │
+└─────────────────────────────────────┘
+```
+
+**On subsequent runs**: The automation runs ~10x faster thanks to caching! Stagehand reuses DOM snapshots from the `credit-karma-cache/` directory instead of re-analyzing every page element.
+
+## COMMON PITFALLS
+
+- **Missing environment variables**: Ensure your `.env` file contains valid `BROWSERBASE_PROJECT_ID` and `BROWSERBASE_API_KEY`. The script will fail silently if these are missing.
+- **Cache directory conflicts**: If you run multiple Credit Karma automations simultaneously, use different `cache_dir` values to avoid conflicts. Example: `cache_dir="credit-karma-cache-user1"`
+- **Stale cache issues**: If Credit Karma's UI changes significantly, delete the `credit-karma-cache/` directory to regenerate fresh snapshots.
+- **Rate limiting**: Credit Karma may rate-limit requests. Use Browserbase's stealth mode (`advanced_stealth=True`) or add delays between runs if you encounter blocks.
+
+Find more information on your Browserbase dashboard → [Sessions](https://browserbase.com/sessions)
+
+## USE CASES
+
+- **Mortgage shopping automation**: Compare refinance rates across different credit scores and loan amounts without manual data entry
+- **Financial data aggregation**: Scrape mortgage rates for market analysis, customer comparison tools, or personal finance apps
+- **Testing and QA**: Validate Credit Karma's UI flows and form handling across different user scenarios
+- **Personal finance tracking**: Monitor how rate changes affect your specific refinance scenario over time
+- **Lead generation tools**: Pre-qualify mortgage leads by automating rate checks based on customer-provided information
+
+## NEXT STEPS
+
+- **Extract rate data**: Use Stagehand's `extract()` method to scrape the mortgage rates table and return structured JSON data
+- **Loop through scenarios**: Iterate through multiple `USER_CONFIG` dictionaries to compare rates across different credit scores or ZIP codes
+- **Add error recovery**: Implement retry logic with exponential backoff for network failures or CAPTCHA challenges
+- **Schedule regular runs**: Set up a cron job or GitHub Action to track rate changes daily/weekly
+- **Integrate with databases**: Store extracted rates in PostgreSQL, MongoDB, or Airtable for historical tracking
+- **Enable proxies**: Set `proxies=True` in `browserbase_session_create_params` to rotate IPs and avoid rate limiting
+
+## HELPFUL RESOURCES
+
+- 📚 [Stagehand Documentation](https://docs.stagehand.dev)
+- 📚 [Browserbase Documentation](https://docs.browserbase.com)
+- 📚 [Caching Guide](https://docs.stagehand.dev/v3/best-practices/caching)
+- 📚 [act() with Variables](https://docs.stagehand.dev/v3/basics/act)
+- 🎮 [Browserbase Dashboard](https://browserbase.com/dashboard)
+- 🎮 [Browserbase Playground](https://www.browserbase.com/playground)
+- 💡 [Try it out: Credit Karma Mortgage Rates](https://www.creditkarma.com/home-loans/mortgage-rates)
+- 📧 [Browserbase Support](mailto:support@browserbase.com)
+- 💬 [Join the Discord](https://discord.gg/browserbase)

python/credit-karma-cache/main.py

@@ -0,0 +1,152 @@
+# Stagehand + Browserbase: Credit Karma Mortgage Rates with Caching - See README.md for full documentation
+
+import asyncio
+import os
+
+from dotenv import load_dotenv
+from pydantic import BaseModel, Field
+
+from stagehand import Stagehand, StagehandConfig
+
+load_dotenv()
+
+# User configuration for mortgage rate lookup
+# Modify these values to customize the mortgage rate search
+USER_CONFIG = {
+    "credit_score": "Above 760",
+    "zipcode": "94109",
+    "loan_balance": "500000",
+    "home_value": "1000000",
+    "cash_out": "200000",
+}
+
+
+class MortgageRate(BaseModel):
+    lender: str = Field(..., description="Name of the lender")
+    interest_rate: str = Field(..., description="Interest rate percentage")
+    apr: str = Field(..., description="APR percentage")
+    monthly_payment: str = Field(..., description="Monthly payment amount")
+
+
+class MortgageRates(BaseModel):
+    rates: list[MortgageRate] = Field(..., description="List of mortgage rate offers")
+
+
+async def main():
+    print("Starting Credit Karma Mortgage Rate Automation...")
+
+    # Initialize Stagehand with Browserbase for cloud-based browser automation.
+    config = StagehandConfig(
+        env="BROWSERBASE",
+        api_key=os.getenv("BROWSERBASE_API_KEY"),
+        project_id=os.getenv("BROWSERBASE_PROJECT_ID"),
+        model_name="google/gemini-2.5-flash",
+        model_api_key=os.getenv("GOOGLE_API_KEY"),
+        verbose=0,
+        # 0 = errors only, 1 = info, 2 = debug
+        # (When handling sensitive data like passwords or API keys, set verbose: 0 to prevent secrets from appearing in logs.)
+        # https://docs.stagehand.dev/configuration/logging
+        browserbase_session_create_params={
+            "project_id": os.getenv("BROWSERBASE_PROJECT_ID"),
+        },
+    )
+
+    try:
+        async with Stagehand(config) as stagehand:
+            print("Stagehand initialized successfully!")
+
+            # Display live view URL for debugging and monitoring
+            session_id = None
+            if hasattr(stagehand, "session_id"):
+                session_id = stagehand.session_id
+            elif hasattr(stagehand, "browserbase_session_id"):
+                session_id = stagehand.browserbase_session_id
+
+            if session_id:
+                print(f"Live View Link: https://browserbase.com/sessions/{session_id}")
+
+            page = stagehand.page
+
+            print("Navigating to Credit Karma mortgage rates page...")
+            await page.goto(
+                "https://www.creditkarma.com/home-loans/mortgage-rates",
+                wait_until="domcontentloaded",
+            )
+
+            await page.act(
+                "click on the 'Refinance' tab button in the mortgage rate calculator form (not a navigation link)"
+            )
+            print("Selected Refinance tab")
+
+            await page.act(
+                f"in the mortgage calculator form, select '{USER_CONFIG['credit_score']}' from the credit score dropdown"
+            )
+            print(f"Selected credit score: {USER_CONFIG['credit_score']}")
+
+            await page.act(
+                f"in the mortgage calculator form, enter '{USER_CONFIG['zipcode']}' in the ZIP code field"
+            )
+            print(f"Entered ZIP code: {USER_CONFIG['zipcode']}")
+
+            await page.act(
+                f"in the mortgage calculator form, enter '{USER_CONFIG['loan_balance']}' in the current loan balance field"
+            )
+            print(f"Entered loan balance: ${USER_CONFIG['loan_balance']}")
+
+            await page.act(
+                f"in the mortgage calculator form, enter '{USER_CONFIG['home_value']}' in the estimated home value field"
+            )
+            print(f"Entered home value: ${USER_CONFIG['home_value']}")
+
+            await page.act(
+                f"in the mortgage calculator form, enter '{USER_CONFIG['cash_out']}' in the cash out amount field"
+            )
+            print(f"Entered cash-out amount: ${USER_CONFIG['cash_out']}")
+
+            await page.act(
+                "click the 'Get my rates' or 'See rates' submit button in the mortgage calculator form"
+            )
+            print("Clicked 'Get my rates' button")
+
+            # Extract mortgage rates using Pydantic schema for structured data
+            mortgage_rates = await page.extract(
+                "Extract all mortgage rate offers shown on the page. For each offer, include: lender name, interest rate, APR, and monthly payment.",
+                schema=MortgageRates,
+            )
+
+            print("\n=== Credit Karma Refinance Query Summary ===")
+            print(f"Credit Score: {USER_CONFIG['credit_score']}")
+            print(f"ZIP Code: {USER_CONFIG['zipcode']}")
+            print(f"Loan Balance: ${USER_CONFIG['loan_balance']}")
+            print(f"Home Value: ${USER_CONFIG['home_value']}")
+            print(f"Cash Out: ${USER_CONFIG['cash_out']}")
+            print("=============================================\n")
+
+            print("=== Mortgage Rate Results ===")
+            if mortgage_rates and mortgage_rates.rates:
+                for rate in mortgage_rates.rates:
+                    print(
+                        f"  {rate.lender}: {rate.interest_rate} ({rate.apr} APR) - {rate.monthly_payment}/mo"
+                    )
+            else:
+                print("No rates found")
+            print("=============================\n")
+
+        print("Session closed successfully")
+
+    except Exception as error:
+        print(f"Error during mortgage rate lookup: {error}")
+        raise
+
+
+if __name__ == "__main__":
+    try:
+        asyncio.run(main())
+    except Exception as err:
+        print(f"Error in credit karma automation: {err}")
+        print("Common issues:")
+        print("  - Check .env file has BROWSERBASE_PROJECT_ID and BROWSERBASE_API_KEY")
+        print("  - Verify GOOGLE_API_KEY is set for the model")
+        print("  - Credit Karma page structure may have changed")
+        print("Docs: https://docs.stagehand.dev/v3/first-steps/introduction")
+        exit(1)

python/credit-karma-cache/pyproject.toml

@@ -0,0 +1,13 @@
+[project]
+name = "credit-karma-cache"
+version = "0.1.0"
+description = "Credit Karma Mortgage Rates automation with Stagehand and Browserbase"
+requires-python = ">=3.10"
+dependencies = [
+    "stagehand>=0.5.0,<1.0.0",
+    "python-dotenv",
+]
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"

python/human-in-the-loop/README.md

@@ -0,0 +1,66 @@
+# Stagehand + Browserbase: Human-in-the-Loop Approval
+
+## AT A GLANCE
+
+- Goal: demonstrate how to pause an automated browser workflow at a decision point, give a human visibility via the live Browserbase session URL, and resume or abort based on their approve/reject input.
+- Flow: navigate to a book product page → extract title/price/rating via Stagehand → evaluate configurable purchase rules → PAUSE if rules triggered → human types "approve" or "reject" in terminal → proceed with Add to basket or abort gracefully.
+
+## GLOSSARY
+
+- human-in-the-loop: a workflow design pattern where automation pauses at a critical decision point and waits for a human to review and approve or reject before continuing.
+- decision gate: the specific point in the workflow where execution is suspended until a human provides input.
+- approval timeout: a safety mechanism that auto-rejects if no human response is received within the configured time limit (default: 2 minutes).
+- books.toscrape.com: a public practice site designed for scraping exercises — no credentials, no rate limits, safe for demos.
+
+## QUICKSTART
+
+1. cd python/human-in-the-loop
+2. pip install -r requirements.txt
+3. playwright install chromium
+4. cp .env.example .env
+5. Add BROWSERBASE_API_KEY, BROWSERBASE_PROJECT_ID, and OPENAI_API_KEY to .env
+6. python main.py
+
+## EXPECTED OUTPUT
+
+- Navigates to a book product page on books.toscrape.com
+- Extracts title, price, rating, and availability using Stagehand
+- Evaluates configurable purchase rules (price > £20 or rating < 3 stars)
+- Because the default book costs £51.77, the price rule triggers and prints a PAUSED banner with the live Browserbase session URL and extracted product data
+- Waits for you to type "approve" or "reject" in the terminal (2-minute timeout)
+- APPROVE: clicks "Add to basket" and confirms success
+- REJECT: aborts gracefully with a message
+- Closes the browser session cleanly
+
+## COMMON PITFALLS
+
+- "ModuleNotFoundError: No module named 'browserbase'": run pip install -r requirements.txt
+- "Executable doesn't exist": run playwright install chromium after pip install
+- Missing credentials: verify .env contains BROWSERBASE_API_KEY, BROWSERBASE_PROJECT_ID, and OPENAI_API_KEY
+- Approval timeout: you have 2 minutes to respond after the PAUSED banner appears; increase APPROVAL_TIMEOUT_SECS in the config block at the top of main.py if needed
+- Book URL changed: books.toscrape.com is a static practice site, but if the specific URL breaks, update BOOK_URL in the config block to any other product page on the site
+- Testing the auto-approve path: set PRICE_THRESHOLD to 100 in the config block — the book at £51.77 will no longer trigger the rule and the workflow will auto-approve without pausing
+
+## USE CASES
+
+- Purchase approval: pause before completing a transaction above a dollar threshold, requiring a manager sign-off before the bot proceeds.
+- Content moderation: pause before posting or publishing content flagged by automated rules, routing it to a human reviewer.
+- Data pipeline review: pause an automated data entry workflow when an extracted value looks anomalous, letting a human verify before the record is committed.
+- Compliance gates: pause before submitting forms on regulated platforms, ensuring a human has reviewed the data before the bot clicks Submit.
+
+## NEXT STEPS
+
+- Replace stdin with a webhook: instead of terminal input, send the decision payload to an HTTP endpoint (Slack, email, or a custom review UI) and poll for the response.
+- Structured decisions: extend the input schema to accept "approve", "reject", or "modify:<new_value>" to allow humans to correct extracted data before resuming.
+- Logging: record every pause event (session ID, extracted data, decision, timestamp) to a database for auditing.
+- Parallel workflows: run multiple product checks concurrently and fan out approval requests, collecting decisions asynchronously.
+
+## HELPFUL RESOURCES
+
+📚 Stagehand Docs: https://docs.stagehand.dev/v3/first-steps/introduction
+🎮 Browserbase: https://www.browserbase.com
+📚 Sessions Docs: https://docs.browserbase.com/features/sessions
+💡 Try it out: https://www.browserbase.com/playground
+🔧 Templates: https://www.browserbase.com/templates
+📧 Need help? support@browserbase.com
+💬 Discord: http://stagehand.dev/discord