Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,8 @@ puts result.issues.to_a.map(&:message)
- `pages`: fetched page snapshots
- `issues`: structured issues with `code`, `severity`, `category`, `url`, and `message`

`result.ok?` returns `false` if any error, warning, or notice is present.
`result.ok?` returns `false` when an error is present. Warnings and notices
remain available through `result.issues` without making the result fail.

## Rails Usage

Expand Down Expand Up @@ -177,7 +178,10 @@ Crawlscope.configuration.audit
```

Rake tasks apply it automatically because `crawlscope/tasks` loads the gem
before Rails evaluates the initializer.
before Rails evaluates the initializer. Each Rake entry point passes the shared
`Crawlscope.configuration` object to the CLI, so configured base URLs, sitemap
paths, registries, and runtime settings are preserved unless a task argument or
environment override replaces them.

Then run:

Expand Down Expand Up @@ -220,7 +224,12 @@ bundle exec rake crawlscope:validate:ldjson URL=https://example.com/article
bundle exec rake 'crawlscope:validate:ldjson[https://example.com/article]'
```

`crawlscope:validate` runs all default sitemap rules: indexability, metadata, structured data, uniqueness, content quality, and links. `URL` is the site base. Without `SITEMAP`, Crawlscope uses `/sitemap.xml`. With `SITEMAP`, Crawlscope uses `URL` as the site base and validates URLs from that sitemap. `SITEMAP` may be a full URL or a local file path.
`crawlscope:validate` runs all default sitemap rules: indexability, metadata,
structured data, uniqueness, content quality, and links. `URL` is the site
base. Without `SITEMAP`, Crawlscope uses the configured sitemap path, then
falls back to `/sitemap.xml`. With `SITEMAP`, Crawlscope uses `URL` as the site
base and validates URLs from that sitemap. `SITEMAP` may be a full URL or a
local file path.

Plain `rake` does not pass `--url` style flags to tasks. Use `URL=...` or the
task-argument form above instead.
Expand Down
4 changes: 2 additions & 2 deletions lib/crawlscope/rake_tasks.rb
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ def validate_rule(rule, url: nil, sitemap_path: nil)
validate(url: url, sitemap_path: sitemap_path, rule_names: rule)
end

def run(command, argv: [])
status = Cli.start([command, *argv], out: $stdout, err: $stderr)
def run(command, argv: [], configuration: Crawlscope.configuration)
status = Cli.start([command, *argv], out: $stdout, err: $stderr, configuration: configuration)
exit(status) unless status.zero?
end

Expand Down
36 changes: 34 additions & 2 deletions test/crawlscope/rake_tasks_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
require "test_helper"

class CrawlscopeRakeTasksTest < Minitest::Test
Rule = Data.define(:code)

def setup
@original_start = Crawlscope::Cli.method(:start)
end
Expand All @@ -13,6 +15,7 @@ def teardown
singleton_class.define_method(:start) do |*args, **kwargs|
original_start.call(*args, **kwargs)
end
Crawlscope.reset!
end

def test_validate_passes_rake_arguments_to_cli
Expand Down Expand Up @@ -56,14 +59,43 @@ def test_ldjson_passes_rake_url_argument_to_cli
)
end

def test_run_passes_the_shared_configuration_to_cli
calls = capture_cli_calls
configuration = Crawlscope.configuration

Crawlscope::RakeTasks.run("validate")

assert_same configuration, calls.fetch(0).fetch(:kwargs).fetch(:configuration)
end

def test_validate_passes_the_host_rule_registry_to_cli
registry = Crawlscope::RuleRegistry.new(rules: [Rule.new(:host)], default_codes: [:host])
Crawlscope.configure { |configuration| configuration.rule_registry = registry }
calls = capture_cli_calls

Crawlscope::RakeTasks.validate

call = calls.fetch(0)
assert_equal "validate", call.fetch(:argv).first
assert_same registry, call.fetch(:kwargs).fetch(:configuration).rule_registry
end

def test_run_exits_with_a_nonzero_cli_status
capture_cli_calls(status: 2)

error = assert_raises(SystemExit) { Crawlscope::RakeTasks.run("validate") }

assert_equal 2, error.status
end

private

def capture_cli_calls
def capture_cli_calls(status: 0)
calls = []
singleton_class = class << Crawlscope::Cli; self; end
singleton_class.define_method(:start) do |argv, **kwargs|
calls << {argv: argv, kwargs: kwargs}
0
status
end
calls
end
Expand Down
Loading