Repository navigation
Mongo document read with non hex objectid #18600
Description
I have a mongo collection which has non-hex '_id' in form a string.
I cant read them into a PCollection getting following exception
Exception in thread "main" java.lang.IllegalArgumentException: invalid hexadecimal representation of an ObjectId: [somestring]
at org.bson.types.ObjectId.parseHexString(ObjectId.java:523)
at org.bson.types.ObjectId.<init>(ObjectId.java:237)
at org.bson.json.JsonReader.visitObjectIdConstructor(JsonReader.java:674)
at org.bson.json.JsonReader.readBsonType(JsonReader.java:197)
at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:139)
at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:45)
at org.bson.codecs.configuration.LazyCodec.decode(LazyCodec.java:47)
at org.bson.codecs.DocumentCodec.readValue(DocumentCodec.java:215)
at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:141)
at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:45)
at org.bson.codecs.DocumentCodec.readValue(DocumentCodec.java:215)
at org.bson.codecs.DocumentCodec.readList(DocumentCodec.java:222)
at org.bson.codecs.DocumentCodec.readValue(DocumentCodec.java:208)
at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:141)
at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:45)
at org.bson.Document.parse(Document.java:105)
at org.bson.Document.parse(Document.java:90)
at org.apache.beam.sdk.io.mongodb.MongoDbIO$BoundedMongoDbReader.start(MongoDbIO.java:472)
at org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$BoundedReadEvaluator.processElement(BoundedReadEvaluatorFactory.java:141)
at org.apache.beam.runners.direct.TransformExecutor.processElements(TransformExecutor.java:146)
at org.apache.beam.runners.direct.TransformExecutor.run(TransformExecutor.java:110)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Imported from Jira BEAM-3165. Original Jira may contain additional context.
Reported by: usopan.
Activity
johnjcasey commentedon Mar 13, 2023
Based on #17084, this looks to be fixed. @y1chi is that the case?
y1chi commentedon Mar 13, 2023
I think the PR is unrelated to this issue. MongoDB IO relies on the default '_id' format, that means user won't be able to explicitly set the reserved '_id' field in the document, but is free to add another field for their own custom document id and index on that.
johnjcasey commentedon Mar 13, 2023
That PR links back to the original jira, so I was confused