Bug #9678

OAI-PMH harvesting and Index issues

Added by Panagiota Koltsida over 2 years ago. Updated over 2 years ago.

Status:ClosedStart date:Sep 13, 2017
Priority:HighDue date:
Assignee:Nikolas Laskaris% Done:

100%

Category:-
Sprint:zz - Index and Search refactoring
Milestones:
Duration:

Description

I have added a data source to be harvested in iSearch VRE using the OAI-PMH portlet. The harvesting seems that is working fine and the number of records are increasing, but nothing is being indexed in the Elastic search.

History

#1 Updated by Nikolas Laskaris over 2 years ago

  • % Done changed from 0 to 80
  • Status changed from New to In Progress

As i see from the production's monitorig tools, the 'faulty' case is of the https://oai.datacite.org/oai repo.
I have added the same source on preprod, and it seems to work fine (it harvests the records, it indexes them and they are immediately available on the search portlet).
I've set up the version of the prod/preprod on our devel machines, and as i see it also works fine there.
I'll try to delete/update/reindex the particular source to see if that brings us in the faulty situation.
I also need to investigate the production logs to see what went wrong in that particular case.

#2 Updated by Nikolas Laskaris over 2 years ago

  • % Done changed from 80 to 100
  • Status changed from In Progress to Closed

After a restarting of the harvesting service, both the harvesting and indexing processes seems to work smoothly, having the records immediately available to the search facilities.
We couldn't reproduce the error, no matter how much we did stress the services after the restart. We consider it closed.

Also available in: Atom PDF