asked on June 1, 2021
Hi,
We're seeing an issue with the new MongoDB connector. Do you have any advice on how to fix this?
2021-06-01T15:57:54.469Z|Flexmonster MongoDB Connector|Quering database Analytics-INT2 to get schema of index Sales 2021-06-01T15:57:54.708Z|Flexmonster MongoDB Connector|Client query: {"members":{"field":{"uniqueName":"Brand"}}} 2021-06-01T15:57:54.709Z|Flexmonster MongoDB Connector|Getting members data 2021-06-01T15:57:54.709Z|Flexmonster MongoDB Connector|Generated pipeline query to MongoDB [{"$project":{"Brand":"$Brand"}},{"$group":{"_id":{"Brand":"$Brand"}}}] /usr/src/app/node_modules/mongodb/lib/core/cursor.js:35 throw err; ^ TypeError: Cannot read property 'length' of null at MongoResponseParser.parseValueFromComplexType (/usr/src/app/node_modules/flexmonster-mongo-connector/build/parsers/MongoResponseParser.js:287:39) at /usr/src/app/node_modules/flexmonster-mongo-connector/build/parsers/MongoResponseParser.js:123:41 at /usr/src/app/node_modules/mongodb/lib/cursor.js:745:11 at handleCallback (/usr/src/app/node_modules/mongodb/lib/core/cursor.js:32:5) at nextFunction (/usr/src/app/node_modules/mongodb/lib/core/cursor.js:867:5) at AggregationCursor._next (/usr/src/app/node_modules/mongodb/lib/core/cursor.js:202:5) at loop (/usr/src/app/node_modules/mongodb/lib/operations/cursor_ops.js:114:10) at each (/usr/src/app/node_modules/mongodb/lib/operations/cursor_ops.js:93:18) at /usr/src/app/node_modules/mongodb/lib/operations/cursor_ops.js:103:7 at /usr/src/app/node_modules/mongodb/lib/utils.js:704:5 npm ERR! code ELIFECYCLE npm ERR! errno 1 npm ERR! flexmonster-proxy-mongo@1.0.0 serve: `node build/server.js` npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the flexmonster-proxy-mongo@1.0.0 serve script. npm ERR! This is probably not a problem with npm. There is likely additional logging output above. npm ERR! A complete log of this run can be found in: npm ERR! /root/.npm/_logs/2021-06-01T15_57_55_945Z-debug.log npm ERR! code ELIFECYCLE npm ERR! errno 1 npm ERR! flexmonster-proxy-mongo@1.0.0 start: `npm run serve` npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the flexmonster-proxy-mongo@1.0.0 start script. npm ERR! This is probably not a problem with npm. There is likely additional logging output above. npm ERR! A complete log of this run can be found in: npm ERR! /root/.npm/_logs/2021-06-01T15_57_56_042Z-debug.log
⋅ June 1, 2021
Hello Jason,
Thank you for providing us with the stack trace of the error.
It helped us a lot in fixing the issue.
We have published a patch with version 2.9.0-1.
Please let us know if the issue is resolved.
Regards,
Dmytro
⋅ June 2, 2021
Hi,
Thanks for the immediate response - this solved the issue and saved us from rolling back our production update!
I'm seeing around a 50% improvement in performance overall.
I wish that MongoDB would allow index use for aggregations where group is the last stage. This would really improve the performance!
⋅ June 2, 2021
Hello Jason,
Thank you for your prompt response.
We are glad to know that the patch has resolved the issue.
As for the indexing, we totally understand and share your wish. Our new release can benefit from the indexing since the aggregation pipeline starts from the "$match" stage. Still, it is not the "$match" stage that takes most of the time. We wish the "$group" stage could be accelerated with indexing or some other approach. Unfortunately, it is not currently available in MongoDB, as you mentioned above.
We have also added an in-memory cache option for storing MongoDB responses. Do you think you can make it work for some of your use cases?
If you would like to find more information about the new MongoDB connector update, please check our blog post: https://www.flexmonster.com/blog/mongodb-pivot-table/.
Regards,
Dmytro
