Skip to content

Commit a73aee4

Browse files
committed
fix
1 parent 1f100b8 commit a73aee4

2 files changed

Lines changed: 24 additions & 32 deletions

File tree

‎docs/source/user_guide/catalog.rst‎

Lines changed: 24 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -169,18 +169,18 @@ it; writers also allow ``WithFileSystemSchemeToIdentifierMap`` to override
169169
file-system selection. For format tables, use ``WriteContextBuilder(FormatTable)``
170170
instead.
171171

172-
A file system passed to ``WithFileSystem`` is used exactly as it is, so a custom
173-
one that signs each request itself stays authenticated by drawing the table's
174-
temporary credentials from ``Catalog::GetTableCredentialProvider``: hold the provider
175-
and call ``CredentialProvider::GetCredentials`` at each access, which returns
176-
credentials that are still valid and reloads them before they expire. A catalog
177-
that issues no per-table credentials returns a null provider, and the custom file
178-
system's own configuration authenticates the access.
172+
A file system passed to ``WithFileSystem`` is used exactly as it is: neither the
173+
built-in file systems nor the table's own credentials are involved, so a custom
174+
one has to authenticate its own accesses. Draw credentials that expire from a
175+
``CredentialProvider`` you build with ``CredentialProviderFactory``: hold the
176+
provider and call ``CredentialProvider::GetCredentials`` at each access, which
177+
returns credentials that are still valid and reloads them before they expire.
179178

180179
.. code-block:: cpp
181180
182181
PAIMON_ASSIGN_OR_RAISE(std::shared_ptr<paimon::CredentialProvider> provider,
183-
shared_catalog->GetTableCredentialProvider(paimon::Identifier("db", "tbl")));
182+
paimon::CredentialProviderFactory::Get("my-token-service",
183+
"oss://bucket/tbl", options));
184184
// MyFileSystem merges provider->GetCredentials() over its own options per access.
185185
auto my_fs = std::make_shared<MyFileSystem>(provider);
186186
write_builder.WithCatalog(shared_catalog, paimon::Identifier("db", "tbl"))
@@ -285,9 +285,8 @@ Both builders read the table's schema from the catalog when ``Finish()`` builds
285285
the context, which is one catalog request; call ``SetTableSchema`` with a schema
286286
you already hold to skip it. The table's file system, including the per-table
287287
temporary credentials a catalog serves through ``Catalog::GetTableFileSystem``,
288-
is used for both. ``WithFileSystem`` overrides it, and a custom file system
289-
that signs its own requests can draw the table's credentials from
290-
``Catalog::GetTableCredentialProvider`` as shown above; the read builder also allows
288+
is used for both. ``WithFileSystem`` overrides it with a file system that is used
289+
as-is and authenticates its own accesses, and the read builder also allows
291290
``WithFileSystemSchemeToIdentifierMap``.
292291

293292
Only the main branch is supported. Both builders reject other branches in the
@@ -367,13 +366,19 @@ tags, branch management and consumers — are not supported.
367366
Authenticating with credentials of your own
368367
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
369368
Credentials that a catalog does not issue — from a token service of your own, or
370-
from a catalog Paimon has no client for — reach the built-in file systems through
371-
a ``CredentialProvider``. Implement it together with a
372-
``CredentialProviderFactory`` that builds it, register the factory with
373-
``REGISTER_PAIMON_FACTORY``, and name it in the ``fs.credential.provider``
374-
option: the file system then builds the provider once and asks it for credentials
375-
as it signs, so credentials are reloaded as they expire without the file system
376-
being rebuilt.
369+
from a store Paimon has no client for — reach a file system through a
370+
``CredentialProvider`` that you consult yourself. Implement it together with a
371+
``CredentialProviderFactory`` that builds it and register the factory with
372+
``REGISTER_PAIMON_FACTORY``. Build a provider with
373+
``CredentialProviderFactory::Get``, wrap it in a ``FileSystem`` of your own that
374+
calls ``CredentialProvider::GetCredentials`` as it signs each access, and pass
375+
that file system in through ``Catalog::Create`` or a builder's ``WithFileSystem``.
376+
A file system passed this way is used as-is, so the credentials are reloaded as
377+
they expire without it being rebuilt.
378+
379+
The built-in file systems — ``oss``, ``s3``, ``local``, ``jindo`` — do not consult
380+
a provider: they sign with the static credentials of their own options. Bring a
381+
file system of your own when the credentials expire.
377382

378383
.. code-block:: cpp
379384
@@ -409,18 +414,6 @@ the factory is called with are those of the accesses to authenticate, which is
409414
where an implementation reads its own configuration, such as the address of the
410415
token service, from.
411416

412-
.. code-block:: none
413-
414-
fs.credential.provider=my-token-service
415-
416-
Only the OSS file system consults a provider today; the others report
417-
``NotImplemented`` when the option names one, rather than signing with the
418-
credentials of the moment they were built and failing once those expire. All
419-
factories share one identifier space, so an identifier a file system factory
417+
All factories share one identifier space, so an identifier a file system factory
420418
already takes — ``oss``, ``s3``, ``local``, ``jindo`` — would replace it; name the
421419
provider after where its credentials come from instead.
422-
423-
This is the other of the two ways a provider is reached. A custom file system
424-
passed to ``WithFileSystem`` signs its own requests and pulls from the provider
425-
itself, which is what ``Catalog::GetTableCredentialProvider`` serves; a built-in
426-
file system signs on your behalf and is pointed at a provider by this option.

‎src/paimon/rest/rest_token_file_system_test.cpp‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,6 @@
3737
#include "paimon/common/utils/checked_cast.h"
3838
#include "paimon/common/utils/scope_guard.h"
3939
#include "paimon/defs.h"
40-
#include "paimon/fs/credential_provider.h"
4140
#include "paimon/fs/local/local_file_system.h"
4241
#include "paimon/rest/mock_rest_server.h"
4342
#include "paimon/rest/rest_api.h"

0 commit comments

Comments
 (0)