asked on January 19, 2023
I'm setting up a proof of concept to make sure your product will work for a project my team is implementing. I just ran into a problem setting up an index with your data server when loading a local JSON file. Below is a sample that illustrates the problem:
[
{
"projectName": "Somewhere in San Antonio",
"projectAddressCity": "San Antonio",
"projectAddressStreetNumber": "10500",
"projectAddressStreetName": "SeaWorld Dr.",
"projectCreatedBy": "Bob the Builder",
"projectCreatedAt": 1673876055523,
"projectUpdatedBy": null,
"projectUpdatedAt": null,
},
{
"projectName": "121 Residential Thing",
"projectAddressCity": "Austin",
"projectAddressStreetNumber": "123",
"projectAddressStreetName": "Lamar Blvd.",
"projectCreatedBy": "Blob the Builder",
"projectCreatedAt": 1664079340869,
"projectUpdatedBy": "Some Admin Guy",
"projectUpdatedAt": 1673962641577
}
]
The error I was getting was: Index _____ cannot be loaded. Details: Cannot load data from JSON file at ".../sample-data/data.json". Please check file format. The requested operation requires an element of type 'String', but the target element has type 'Number'.
The problem is that the very first record has a null value for the projectUpdatedAt property (a timestamp).
Your data server assumes this property is a string. When the second record is parsed, it fails with the error above. If I supply a numeric value for the first record's projectUpdatedAt the problem goes away.
Is there some way to feed a JSON schema document to your data server so it knows what to expect so it does't have to guess the data type of a null value?
It was somewhat challenging to sort out what was going wrong because your error message was relatively generic and didn't identify the field that was causing the problem.
If it matters, I'm running on a Mac with Apple Silicon.
⋅ January 20, 2023
Hello, Justin!
Thank you for contacting us.
Kindly note that in Flexmonster, data types are automatically defined using the values from the first object of JSON data.
Keeping this in mind, we kindly suggest avoiding null data in the first JSON object.
Hope you will find our answer helpful.
Feel free to contact us in any other questions arise.
Regards,
Solomiia
⋅ January 20, 2023
Thank-you for the follow up Solomiia. That's not the answer I was hoping for as I have to deal with whatever data flows my way. I'll have come up with a way to ensure the first record is a complete one even though that's not how our system works.
Perhaps you could pass along the suggestion to your development team that having the ability to pair a json schema document with a data server index would be the most reliable and straight forward way to solve this particular problem.
cheers
Justin
⋅ January 20, 2023
Hello again, Justin!
Thank you for getting back to us.
We understand that the described parsing behavior has an impact on your user experience.
We have already discussed the described case with our development team and will provide a fix with the ETA February 6th.
In addition to that, we already have in our customers' wishlist the task of adding a JSON schema when loading files in Flexmonster Data Server. We have noted that your company is interested in mentioned feature and will notify you about any updates on the matter.
Our team will notify you when the fix is ready.
You are welcome to use the workaround mentioned in the previous message till the fix is ready.
Feel free to contact us in case of any other questions.
Regards,
Solomiia
⋅ January 20, 2023
Thx for the update Solomiia. A+ for rockstar support. Your responses have definitely been affirming for me as a buyer. Much appreciated.
⋅ January 23, 2023
Hello, Justin!
Thank you for your kind words.
Our team will notify you as soon as the fix is ready.
Do not hesitate to contact us if any other questions arise.
Kind regards,
Solomiia
⋅ February 7, 2023
Hello, Justin!
Hope you are doing well.
Our team is happy to inform you that the issue with number columns being treated as strings when the first record is null was fixed.
Now, the Data Server is auto-recognizing the data type based on the first 10 rows of data ignoring the null values.
The fix is available in the 2.9.44 version of Flexmonster: https://www.flexmonster.com/release-notes/version-2-9-44/.
You are welcome to update the component. Here is our updating guide for reference: https://www.flexmonster.com/doc/updating-to-the-latest-version/.
Please let us know if the fix works well for your case.
Looking forward to hearing your feedback.
Regards,
Solomiia
⋅ February 14, 2023
Hello, Justin!
Hope you are having a great week.
Our team is wondering if you had a chance to test the fix with data recognition in Flexmonster Data Server.
Could you please let us know if the fix works well for you?
Looking forward to hearing from you.
Regards,
Solomiia
⋅ February 14, 2023
Thank-you for the follow up Solomiia. I can't confidently say that scanning the first 10 records is a reliable fix. The data I send to the data server is a result of user activity which I don't control. Scanning 10 records is certainly better than 1 but isn't something I can count on.
What I did implement is the new Mapping functionality. That seems to be the reliable way to ensure that all my date-time fields get indexed correctly, regardless of how many nulls are in the data. So yes, I think my original issue is solved. Thank your team for me!
I'm currently working on a proof of concept that is somewhat challenging to implement. In the system I'm attempting to build reporting for, we have a low level entity with known fields but also dynamic ones. This means I need to build a user interface where users can select the dynamic fields they want to include in their index, then programatically update flexmonster-config.json and restart the data server. I'm about 50% done building all this functionality into a usable Docker image. I'll hopefully have the proof of concept working this week. Once I have that up, my client can try it out and verify that this solution meets their needs.
⋅ February 14, 2023
Hello, Justin!
Thank you for your detailed feedback.
Our team is glad to hear that the new mapping feature works well for you. We'll be waiting for the following updates on the case.
Looking forward to hearing from you.
Regards,
Solomiia
⋅ February 22, 2023
Hello, Justin!
Hope you are doing well.
Our team is wondering if there are any updates on the case.
Do not hesitate to contact us if any further questions arise.
We also want to mention that starting from version 2.9.45 the Mapping in Data Server can be configured using Admin Panel UI tool as well as inside the flexmonster-config.json file. You can read more about the update here: https://www.flexmonster.com/release-notes/version-2-9-45/.
Looking forward to hearing from you.
Regards,
Solomiia
⋅ February 22, 2023
You are definitely on top of customer service! As I mentioned above, I'm working in a server environment where there is no UI available. I had to program my own way of updating flexmonster-config.json and restarting the data server.
Now that I've got that built, my client needs a bit of time to evaluate that Flexmonster will meet their needs. Things are moving a bit slowly on their side, so I suspect that will be 2 - 4 weeks.
I do have it all running in Docker on AWS. I'm pretty happy with how it all came together.
⋅ February 22, 2023
Hello, Justin!
Thank you for your kind words.
Your feedback is valuable to our team.
You are welcome to contact us in case of any other questions.
Kind regards,
Solomiia
