asked on May 31, 2022
Hello FM Team,
We have hosted FM Dataserver webapi app in azure and display data in dataview (FMDataGrid) of angular app. We have set DataRefreshTime setting of DataStorageOptions configuration to 24hrs (1440mins) to reload data from database, as new data gets loaded into database once received from client either weekly/monthly/quarterly based on different clients.
Currently, for testing, we are inserting data into database and we have to restart FM Dataserver Web app service manually to reload indexes and data. is there anyway available such as hitting an endpoint of FMDataServer and reload data, so that new data will be visible in angular app FMdatagrid(dataview), whenever new data get uploaded in database???... If it possible, we can event trigger in ADF/AAS to hit the endpoint and reload data as soon as new data available.
Let us know.
Thanks & Regards,
Shahul
⋅ June 1, 2022
Hi FM Support Team,
Were you able to check on the Above request . We are paid license user and we have a requirement to do a FM dataserver cache refresh as as soon as the ETL data load Jobs get completed instead of a scheduled refresh . Please let us know if there is an Option to do the same . Thanks in advance .
Regds,
Aparna
⋅ June 1, 2022
Hello,
Thank you for reaching out to us.
The general idea of implementing the manual index refresh lies in the following:
Flexmonster Data Server DLL provides the IDataStorage interface:
public interface IDataStorage
{
public Task<IDataStructure> GetOrAddAsync(string cacheKey);
public void Remove(string key);
}
Using IDataStorage, you could create a custom service for reloading indexes in your application. This would allow you to refresh indexes when necessary, not just every X minutes.
Below is a code snippet showing a sample implementation of a custom index reloading service:
public interface IReloadService
{
Task Reload(string indexName);
}
public class ReloadService : IReloadService
{
private readonly IDataStorage _dataStorage;
public ReloadService(IDataStorage dataStorage)
{
_dataStorage = dataStorage;
}
public async Task Reload(string indexName)
{
// Refresh the data index
_dataStorage.Remove(indexName);
await _dataStorage.GetOrAddAsync(indexName);
}
}
Then you could register the service in your application, for example, in Startup.cs:
public IConfiguration Configuration { get; }
public void ConfigureServices(IServiceCollection services)
{
// other services configurations
services.ConfigureFlexmonsterOptions(Configuration);
services.AddFlexmonsterApi();
services.AddScoped<IReloadService, ReloadService>();
// other services configurations
}
After these steps, you will be able to use Reload task inside a custom endpoint, responsible for manual data refresh.
Hope you will find this information helpful.
Best Regards,
Maksym
⋅ June 2, 2022
Hi Maksym,
Thank you. We will implement above suggested solution and let you know, this suffice our need or not.
Thank you!!.
Regards,
Shahul
⋅ June 2, 2022
Hi, Shahul!
We are looking forward to hearing about the result of your implementation.
As always, feel free to contact us if any other questions arise.
Best Regards,
Maksym
⋅ June 3, 2022
Hi Maksym,
One of our developer tried above suggested process/code. Services.AddScoped call throws below error. We tried by changing it to services.AddTransient..result is same. let us know, in reloadservice constructor , anything needed to be passed or any other fix. Error for your reference
Additional note : methods inside interfaces are public
System.AggregateException
HResult=0x80131500
Message=Some services are not able to be constructed (Error while validating the service descriptor 'ServiceType: FlexmonsterDataServer.Models.IReloadService Lifetime: Scoped ImplementationType: FlexmonsterDataServer.Services.ReloadService': Unable to resolve service for type 'FlexmonsterDataServer.Models.IDataStorage' while attempting to activate 'FlexmonsterDataServer.Services.ReloadService'.)
Source=Microsoft.Extensions.DependencyInjection
StackTrace:
at Microsoft.Extensions.DependencyInjection.ServiceProvider..ctor(IEnumerable`1 serviceDescriptors, IServiceProviderEngine engine, ServiceProviderOptions options)
at Microsoft.Extensions.DependencyInjection.ServiceCollectionContainerBuilderExtensions.BuildServiceProvider(IServiceCollection services, ServiceProviderOptions options)
at Microsoft.Extensions.DependencyInjection.DefaultServiceProviderFactory.CreateServiceProvider(IServiceCollection containerBuilder)
at Microsoft.Extensions.Hosting.Internal.ServiceFactoryAdapter`1.CreateServiceProvider(Object containerBuilder)
at Microsoft.Extensions.Hosting.HostBuilder.CreateServiceProvider()
at Microsoft.Extensions.Hosting.HostBuilder.Build()
at FlexmonsterDataServer.Program.Main(String[] args) in D:\Project\Source Code\BIT App FMDataServer\Program.cs:line 18
This exception was originally thrown at this call stack:
Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.CreateArgumentCallSites(System.Type, System.Type, Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteChain, System.Reflection.ParameterInfo[], bool)
Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.CreateConstructorCallSite(Microsoft.Extensions.DependencyInjection.ServiceLookup.ResultCache, System.Type, System.Type, Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteChain)
Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.TryCreateExact(Microsoft.Extensions.DependencyInjection.ServiceDescriptor, System.Type, Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteChain, int)
Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.GetCallSite(Microsoft.Extensions.DependencyInjection.ServiceDescriptor, Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteChain)
Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.ValidateService(Microsoft.Extensions.DependencyInjection.ServiceDescriptor)
Inner Exception 1:
InvalidOperationException: Error while validating the service descriptor 'ServiceType: FlexmonsterDataServer.Models.IReloadService Lifetime: Scoped ImplementationType: FlexmonsterDataServer.Services.ReloadService': Unable to resolve service for type 'FlexmonsterDataServer.Models.IDataStorage' while attempting to activate 'FlexmonsterDataServer.Services.ReloadService'.
Inner Exception 2:
InvalidOperationException: Unable to resolve service for type 'FlexmonsterDataServer.Models.IDataStorage' while attempting to activate 'FlexmonsterDataServer.Services.ReloadService'.
Regards,
Shahul
⋅ June 3, 2022
Hello, Shahul!
Thank you for your response.
We have prepared an example with manual index refreshing. Please check it out on our GitHub: https://github.com/flexmonster/flexmonster-data-server-dll/tree/feature-manual-index-refresh.
We kindly suggest comparing our sample to your project as it seems that mentioned error message appears in the way you embed ReloadService in your project.
Here are some steps we have done to implement manual refreshing:
1. On the server-side, add /refresh HTTP request in FlexmonsterAPIController, which triggers Reload() method of the ReloadService.
The ReloadService is implemented as we have mentioned in our previous answer.
[Route("refresh")]
[HttpGet]
public async Task<IActionResult> RefreshIndex([FromQuery] string index)
{
await _reloadService.Reload(index);
return new JsonResult("");
}
2. On the client-side, add a button onclick eventListener via JQuery in site.js, which refreshes the index and updates pivot data after refreshing complete:
$("#reloadButton").on("click", function () {
$.ajax({
url: "http://localhost:51718/api/refresh?index=" + report.dataSource.index,
type: "get",
success: function () {
pivot.updateData(report.dataSource);
}
});
3. In the appsettings.json set the DataRefreshTime to 0, so the index can be refreshed only manually:
"DataStorageOptions": {
"DataRefreshTime": 0
}
We hope it helps.
Feel free to ask if any other questions arise.
Regards,
Solomiia
⋅ June 7, 2022
Hi Solomiia,
Thanks for your response. We have implemented it and trying to call through automated trigger, whenever clients data get loaded through ADF pipelines. For failsafe option, we need notification on successful/Failure of index reload.
Is there any default/inbuilt notification service (like email) available in FM Dataserver dll??.
Lets us know.
Regards,
Shahul
⋅ June 8, 2022
Hi, Shahul!
Thank you for your reply.
Currently, Flexmonster Data Server DLL does not have built-in services for notifying about the success or failure of index refresh. We suggest implementing this inside the Reload task of the ReloadService or "refresh" controller.
Please let us know if any questions arise.
Best Regards,
Maksym
⋅ June 9, 2022
Hi Maksym,
Thanks for your response. We are implementing email notification service post reload index service endpoint execution to notify admins about success or failure status.
Regards,
Shahul
⋅ June 9, 2022
Hi, Shahul!
Thank you for sharing these details with us. Please let us know in case of any other questions.
Regards,
Maksym