Bug #3253

Couchdb does not support documents with object key starting with _

Added by Luca Frosini over 3 years ago. Updated over 3 years ago.

Status:ClosedStart date:Apr 13, 2016
Priority:UrgentDue date:Apr 15, 2016
Assignee:Luca Frosini% Done:

100%

Category:accounting
Sprint:zz - Accounting Improvement & Fixes
Milestones:
Duration: 3

Description

Fix Job and Task Usage Records model. Remove the _ from variable starting with it.

Please also allows the new model to recover old usage record have underscored variable moving to the new name.


Related issues

Related to gCube - Release #2449: org.gcube.accounting.accounting-lib.2-2-0 Released Mar 01, 2016
Related to gCube - Release #3175: org.gcube.accounting.accounting-lib.2-2-0 (gCore) Released Apr 11, 2016
Related to gCube - Release #3174: document-store-lib.1.1.0 Released Apr 11, 2016
Related to gCube - Release #3178: document-store-lib.1.1.0 (gCore) Released Apr 11, 2016

History

#1 Updated by Luca Frosini over 3 years ago

  • Status changed from New to In Progress

#2 Updated by Luca Frosini over 3 years ago

  • % Done changed from 0 to 80

#3 Updated by Luca Frosini over 3 years ago

  • Related to Release #2449: org.gcube.accounting.accounting-lib.2-2-0 added

#4 Updated by Luca Frosini over 3 years ago

  • Related to Release #3175: org.gcube.accounting.accounting-lib.2-2-0 (gCore) added

#5 Updated by Luca Frosini over 3 years ago

  • Status changed from In Progress to Paused

#6 Updated by Luca Frosini over 3 years ago

  • Sprint changed from zz - Accounting NoSQL Comparison to zz - Accounting Improvement & Fixes

#7 Updated by Luca Frosini over 3 years ago

  • % Done changed from 80 to 100
  • Status changed from Paused to In Progress

#8 Updated by Luca Frosini over 3 years ago

  • Status changed from In Progress to Resolved

#9 Updated by Luca Frosini over 3 years ago

#10 Updated by Luca Frosini over 3 years ago

  • Related to Release #3178: document-store-lib.1.1.0 (gCore) added

#11 Updated by Luca Frosini over 3 years ago

@roberto.cirillo@isti.cnr.it I have tested it using the example record you provided me.

{scope=/d4science.research-infrastructures.eu/gCubeApps, jobId=6847082, _wallDuration=15952, jobName=BIONYM_LOCAL, consumerId=wps.statisticalmanager, id=a9229298-bd0a-43d2-be37-d5a026f405c5, jobEndTime=1457460157466, operationResult=SUCCESS, jobStartTime=1457460141514, jobQualifier=6847082, creationTime=1457460141464, recordType=JobUsageRecord}

Can we try to replace the library on one of the statistical instances?

#12 Updated by Luca Frosini over 3 years ago

  • Status changed from Resolved to Closed

#13 Updated by Luca Frosini over 3 years ago

  • Related to Project Task #700: T10.3 Federated Resources Management [Months: 1-29] added

#14 Updated by Luca Frosini over 3 years ago

  • Related to deleted (Project Task #700: T10.3 Federated Resources Management [Months: 1-29])

#15 Updated by Roberto Cirillo over 3 years ago

  • % Done changed from 100 to 80
  • Priority changed from High to Urgent
  • Status changed from Closed to In Progress

I've installed the new ghn distribution on statistical-manager.d.d4science.org but I see again the same exception:

2016-05-20 17:14:42,870 TRACE persistence.PersistenceBackend [pool-7-thread-1,accountWithFallback:95] Going to account {scope=/gcube/de
vsec/devVRE, jobId=205766, _wallDuration=658441, jobName=ICHTHYOP_MODEL_ONE_BY_ONE, consumerId=julien.barde, id=32fb9d9d-ec01-4257-baf6
-d0fb4abd8fd6, jobEndTime=1463653934809, operationResult=SUCCESS, jobStartTime=1463653276368, wallDuration=658441, jobQualifier=205766,
 creationTime=1463653276254, recordType=JobUsageRecord} using PersistenceCouchDB : org.gcube.documentstore.persistence.PersistenceCouch
DB@2b309389
2016-05-20 17:14:42,877 ERROR persistence.PersistenceBackend [pool-7-thread-1,accountWithFallback:101] {scope=/gcube/devsec/devVRE, job
Id=205766, _wallDuration=658441, jobName=ICHTHYOP_MODEL_ONE_BY_ONE, consumerId=julien.barde, id=32fb9d9d-ec01-4257-baf6-d0fb4abd8fd6, j
obEndTime=1463653934809, operationResult=SUCCESS, jobStartTime=1463653276368, wallDuration=658441, jobQualifier=205766, creationTime=14
63653276254, recordType=JobUsageRecord} was not accounted succesfully from PersistenceCouchDB. Trying to use FallbackPersistenceBackend
.
java.lang.Exception: error contacting couch Db: response code is 500
        at org.gcube.common.couchdb.connector.HttpCouchClient.checkStatus(HttpCouchClient.java:188)
        at org.gcube.common.couchdb.connector.HttpCouchClient.put(HttpCouchClient.java:146)
        at org.gcube.documentstore.persistence.PersistenceCouchDB.createItem(PersistenceCouchDB.java:56)
        at org.gcube.documentstore.persistence.PersistenceCouchDB.reallyAccount(PersistenceCouchDB.java:65)
        at org.gcube.documentstore.persistence.PersistenceBackend.accountWithFallback(PersistenceBackend.java:96)
        at org.gcube.documentstore.persistence.PersistenceBackendMonitor.elaborateFile(PersistenceBackendMonitor.java:51)
        at org.gcube.documentstore.persistence.PersistenceBackendMonitor.elaborateFallbackFile(PersistenceBackendMonitor.java:102)
        at org.gcube.documentstore.persistence.PersistenceBackendMonitor.run(PersistenceBackendMonitor.java:121)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

with these libraries:

  • accounting-lib-2.2.0-3.11.0-128110.jar
  • document-store-lib-couchdb-1.0.1-3.11.0-125796.jar
  • document-store-lib-1.1.0-3.11.0-128109.jar

please investigate

#16 Updated by Luca Frosini over 3 years ago

  • % Done changed from 80 to 100
  • Status changed from In Progress to Feedback

The library is the correct version using wallDuration instead of _wallDuration. This problem is related to retry of fallback.
To solve this we should replace all occurrences in fallback files.

#17 Updated by Roberto Cirillo over 3 years ago

  • Status changed from Feedback to Resolved

I've replaced the fallback log files with "_wallDuration" field. Now it seems to work well on statistical-manager.d.

#18 Updated by Roberto Cirillo over 3 years ago

  • Status changed from Resolved to Closed

#19 Updated by Luca Frosini over 3 years ago

  • Related to Project Task #700: T10.3 Federated Resources Management [Months: 1-29] added

#20 Updated by Luca Frosini over 3 years ago

  • Related to deleted (Project Task #700: T10.3 Federated Resources Management [Months: 1-29])

Also available in: Atom PDF