asked on September 3, 2025
So im using custom data api server with mongo connector, issue is in the backend’s getSchema function in /fields endpoint which is implemented by calling findOne() on a MongoDB collection to infer available fields. Since findOne() only returns a single arbitrary document, there is a high likelihood that not all possible fields in the collection will be represented. This leads to incomplete schema detection, which can cause UI components relying on /fields to display incomplete data
To make getSchema more efficient we can use $sample with let say 10 size, Although it doesnt guarantee 100% schema coverage if fields are very sparse, but it'll be more representative that findOne(), also $sample was introduced in 3.x.x so there will be no compatability issue
⋅ September 4, 2025
Hello,
Thank you for your suggestion.
You are right that relying on the findOne() for retrieving fields can lead to incomplete results when documents do not consistently contain all fields, and there is a potential for achieving more representative results. That said, our connector was designed with a more generic use case in mind, assuming that all fields would be available in the data and represented by null values or empty strings when they are not present.
If you would like to adapt this to your scenario, you can modify the /fields controller in the connector directly: https://github.com/flexmonster/pivot-mongo/blob/master/src/controller/mongo.ts#L59
You can either add a custom query for fields or hard-code the fields corresponding to indexes there.
Please let us know if this approach would work for you.
Best Regards,
Maksym
