Skip to content

ImportError: cannot import name 'Client' from scrapegraph_py when using scrapegraph-py 2.1.0 #1097

Closed

Description

@adithyanajay

Describe the bug

The cognee-community-tasks-scrapegraph package is not compatible with the latest scrapegraph-py (v2.1.0).

When importing scrape_urls from cognee_community_tasks_scrapegraph, the import fails with:

ImportError: cannot import name 'Client' from 'scrapegraph_py'

The package appears to expect the older scrapegraph-py API (Client), while the latest SDK (v2.1.0) no longer exposes that class.


To Reproduce

  1. Create a new Python virtual environment.

  2. Install:

    pip install cognee-community-tasks-scrapegraph scrapegraph-py
  3. Create a simple script:

    import asyncio
    import os
    from cognee_community_tasks_scrapegraph import scrape_urls
    
    os.environ["SGAI_API_KEY"] = "your-api-key"
    
    async def main():
        results = await scrape_urls(
            urls=[
                "https://cognee.ai",
                "https://docs.cognee.ai",
            ],
            prompt="Extract the product description, key features, and target use cases",
        )
    
        print(results)
    
    asyncio.run(main())
  4. Run the script.


Actual behavior

The script fails during import with:

ImportError: cannot import name 'Client' from 'scrapegraph_py'

Traceback points to:

from scrapegraph_py import Client

inside cognee_community_tasks_scrapegraph/scrapegraph_task.py.


Expected behavior

The package should either:

  • support the latest scrapegraph-py SDK (v2.x), or
  • specify a compatible version constraint (for example, pinning scrapegraph-py<2 if only the older API is supported).

Screenshots

Image

Desktop

  • OS: Ubuntu 24.04 LTS
  • Python: 3.12
  • cognee-community-tasks-scrapegraph: 0.1.1
  • scrapegraph-py: 2.1.0

Smartphone

Not applicable.


Additional context

Installed package versions:

cognee-community-tasks-scrapegraph==0.1.1
scrapegraph-py==2.1.0

It appears that scrapegraph_task.py still imports:

from scrapegraph_py import Client

However, Client is no longer available in scrapegraph-py v2.x, resulting in an immediate ImportError before any user code can execute.

Activity

  1. dosubot commented on Jul 1, 2026

    @dosubot

    Hi @adithyanajay! I'm Dosu and I'm helping the ScrapeGraphAI, Inc team.

    The Client class still exists in scrapegraph-py v2.x — it was not removed, but its methods were significantly refactored in PR #1058. Specifically, in v2.x the API changed as follows:

    v1 Method v2 Method
    smartscraper(website_url=, user_prompt=) extract(url=, prompt=, output_schema=)
    searchscraper(user_prompt=) search(query=)
    markdownify(website_url=) scrape(url=)
    get_credits() credits()

    So from scrapegraph_py import Client should still work with scrapegraph-py==2.1.0. If you're getting an ImportError, it's possible there's a packaging or installation conflict in your environment.

    That said, the cognee-community-tasks-scrapegraph package is not part of this repository — it's a third-party community package. The incompatibility you're experiencing is in that external package's code, not in scrapegraph-py itself. The fix would need to come from the maintainers of cognee-community-tasks-scrapegraph.

    As a workaround, you could try pinning to an older version:

    pip install "scrapegraph-py<2"

    Or, if you don't need the cognee-community-tasks-scrapegraph wrapper, you can use scrapegraph-py v2.x directly [1]:

    from scrapegraph_py import Client
    
    client = Client(api_key="your-api-key")
    result = client.extract(url="https://cognee.ai", prompt="Extract the product description, key features, and target use cases")

    To reply, just mention @dosu.


    Docs are dead. Just use Dosu.

    Leave Feedback Ask Dosu about Scrapegraph-ai Share Dosu with your team

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions