Task #5568

Please Update orientdb version to 2.2.12

Added by Luca Frosini about 3 years ago. Updated about 3 years ago.

Status:ClosedStart date:Oct 26, 2016
Priority:NormalDue date:
Assignee:_InfraScience Systems Engineer% Done:

100%

Category:Application
Sprint:Dev Resource Registry Deployment
Infrastructure:Development
Milestones:
Duration:

Description

I already updated the version on ansible script.


Related issues

Related to D4Science Infrastructure - Task #3475: Add 2 OrientDB instances Closed Sep 29, 2016
Related to gCube - Task #12389: Investigate a different configuration for orientdb cluster Closed Sep 05, 2018

History

#1 Updated by Luca Frosini about 3 years ago

  • Related to Task #3475: Add 2 OrientDB instances added

#2 Updated by Luca Frosini about 3 years ago

  • Description updated (diff)

#3 Updated by Luca Frosini about 3 years ago

  • % Done changed from 0 to 50
  • Status changed from New to In Progress
  • Description updated (diff)

thanks to @andrea.dellamico@isti.cnr.it indication I run the upgrade

#4 Updated by Luca Frosini about 3 years ago

I experiencing some issues on orientdb:

  • Only orientdb02-d-d4s was started.
  • trying starting the service manually I obtain bash: ../log/orientdb.log: Permission denied. IMHO is related to the way to start the server in /etc/init.d/orientdb script.
su $ORIENTDB_USER -c "cd \"$ORIENTDB_DIR/bin\"; /usr/bin/nohup ./server.sh 1>../log/orientdb.log 2>../log/orientdb.err &"
  • logs are not available in the configured folder (i.e. ~/logs which is a symbolic link to /var/log/orientdb ). I only have the old logs from previous instance
  • Some variable on json are upper case True instead of true in default-distributed-db-config.json
  • I'm not sure if the distributed server is running. We have forced distributed mode by setting in tag. I don't understadn from documentation if we have to use dserver.sh to start it. So that we could try to use dserver.sh in /etc/init.d/orientdb instead of server.sh.

Here is the content of some files:

cat ~/orientdb-community/config/default-distributed-db-config.json 
{
  "autoDeploy": True,
  "readQuorum": 1,
  "writeQuorum": "majority",
  "executionMode": "undefined",
  "readYourWrites": True,
  "servers": {
    "*": "master"
  },
  "clusters": {
    "internal": {
    },
    "*": {
      "servers": ["<NEW_NODE>"]
    }
  }
}
cat ~/orientdb-community/bin/dserver.sh 
#!/bin/sh
#
# Copyright (c) Orient Technologies LTD (http://www.orientechnologies.com)
#

"${0%dserver.sh}server.sh" -Ddistributed=true $*
cat /etc/init.d/orientdb 
#!/bin/bash
# OrientDB service script
#
# Copyright (c) Orient Technologies LTD (http://www.orientechnologies.com)

# chkconfig: 2345 20 80
# description: OrientDb init script
# processname: orientdb.sh

# You have to SET the OrientDB installation directory here
if [ -f /etc/default/orientdb ] ; then
    . /etc/default/orientdb
else
    ORIENTDB_DIR="/home/orientdb/orientdb-community"
    ORIENTDB_USER="orientdb"
fi

usage() {
    echo "Usage: `basename $0`: <start|stop|status>"
    exit 1
}

start() {
    status
    if [ $PID -gt 0 ]
    then
        echo "OrientDB server daemon was already started. PID: $PID"
        return $PID
    fi
    echo "Starting OrientDB server daemon..."
    cd "$ORIENTDB_DIR/bin"
    su $ORIENTDB_USER -c "cd \"$ORIENTDB_DIR/bin\"; /usr/bin/nohup ./server.sh 1>../log/orientdb.log 2>../log/orientdb.err &"
}

stop() {
    status
    if [ $PID -eq 0 ]
    then
        echo "OrientDB server daemon is already not running"
        return 0
    fi
    echo "Stopping OrientDB server daemon..."
    cd "$ORIENTDB_DIR/bin"
    su $ORIENTDB_USER -c "cd \"$ORIENTDB_DIR/bin\"; /usr/bin/nohup ./shutdown.sh 1>>../log/orientdb.log 2>>../log/orientdb.err &"
}

status() {
    PID=` ps auxw | grep 'orientdb.www.path' | grep java | grep -v grep | awk '{print $2}'`
    if [ "x$PID" = "x" ]
    then
        PID=0
    fi

    # if PID is greater than 0 then OrientDB is running, else it is not
    return $PID
}

if [ "x$1" = "xstart" ]
then
    start
    exit 0
fi

if [ "x$1" = "xstop" ]
then
    stop
    exit 0
fi

if [ "x$1" = "xstatus" ]
then
    status
    if [ $PID -gt 0 ]
    then
        echo "OrientDB server daemon is running with PID: $PID"
        exit 0
    else
        echo "OrientDB server daemon is NOT running"
        exit 3
    fi
fi

usage

#5 Updated by Pasquale Pagano about 3 years ago

  • Tracker changed from Support to Task

#6 Updated by Luca Frosini about 3 years ago

I fixed /etc/init.d/orientdb script on orientdb01-d-d4s like this:

cat /etc/init.d/orientdb 
#!/bin/bash
# OrientDB service script
#
# Copyright (c) Orient Technologies LTD (http://www.orientechnologies.com)

# chkconfig: 2345 20 80
# description: OrientDb init script
# processname: orientdb.sh

# You have to SET the OrientDB installation directory here
if [ -f /etc/default/orientdb ] ; then
    . /etc/default/orientdb
else
    ORIENTDB_DIR="/home/orientdb/orientdb-community"
    ORIENTDB_USER="orientdb"
fi

usage() {
    echo "Usage: `basename $0`: <start|stop|status>"
    exit 1
}

start() {
    status
    if [ $PID -gt 0 ]
    then
        echo "OrientDB server daemon was already started. PID: $PID"
        return $PID
    fi
    echo "Starting OrientDB server daemon..."
    cd "$ORIENTDB_DIR/bin"
    su $ORIENTDB_USER -c "cd \"$ORIENTDB_DIR/bin\"; /usr/bin/nohup ./server.sh 1>/home/orientdb/logs/orientdb.log 2>/home/orientdb/logs/orientdb.err &"
}

stop() {
    status
    if [ $PID -eq 0 ]
    then
        echo "OrientDB server daemon is already not running"
        return 0
    fi
    echo "Stopping OrientDB server daemon..."
    cd "$ORIENTDB_DIR/bin"
    su $ORIENTDB_USER -c "cd \"$ORIENTDB_DIR/bin\"; /usr/bin/nohup ./shutdown.sh 1>>/home/orientdb/logs/orientdb.log 2>>/home/orientdb/logs/orientdb.err &"
}

status() {
    PID=` ps auxw | grep 'orientdb.www.path' | grep java | grep -v grep | awk '{print $2}'`
    if [ "x$PID" = "x" ]
    then
        PID=0
    fi

    # if PID is greater than 0 then OrientDB is running, else it is not
    return $PID
}

if [ "x$1" = "xstart" ]
then
    start
    exit 0
fi

if [ "x$1" = "xstop" ]
then
    stop
    exit 0
fi

if [ "x$1" = "xstatus" ]
then
    status
    if [ $PID -gt 0 ]
    then
        echo "OrientDB server daemon is running with PID: $PID"
        exit 0
    else
        echo "OrientDB server daemon is NOT running"
        exit 3
    fi
fi

usage

Now the server start but I have the following error on ~/logs/orientdb.err

 cat ~/logs/orientdb.err
./server.sh: 108: ./server.sh: cannot create /home/orientdb/orientdb-community/bin/orient.pid: Permission denied

2016-10-26 15:03:43:696 INFO  Loading configuration from: /home/orientdb/orientdb-community/config/orientdb-server-config.xml... [OServerConfigurationLoaderXml]
2016-10-26 15:03:44:336 INFO  OrientDB Server v2.2.12-SNAPSHOT (build 2.2.x@r2a76ee1efc4487d9dc5f43e1ddc2fba1c5a0167c; 2016-10-20 13:55:08+0000) is starting up... [OServer]
2016-10-26 15:03:44:350 INFO  Databases directory: /home/orientdb/databases [OServer]
2016-10-26 15:03:44:486 WARNI Not enough physical memory available for DISKCACHE: 2,938MB (heap=1,820MB direct=524,288MB). Set lower Maximum Heap (-Xmx setting on JVM) and restart OrientDB. Now running with DISKCACHE=256MB [orientechnologies]
2016-10-26 15:03:44:487 INFO  OrientDB config DISKCACHE=256MB (heap=1,820MB direct=524,288MB os=2,938MB) [orientechnologies]
2016-10-26 15:03:45:236 INFO  Listening binary connections on 0.0.0.0:2424 (protocol v.36, socket=default) [OServerNetworkListener]
2016-10-26 15:03:45:244 INFO  Listening http connections on 0.0.0.0:2480 (protocol v.10, socket=default) [OServerNetworkListener]
2016-10-26 15:03:45:270 INFO  Installing dynamic plugin 'studio-2.2.zip'... [OServerPluginManager]
2016-10-26 15:03:45:281 INFO  ODefaultPasswordAuthenticator is active [ODefaultPasswordAuthenticator]
2016-10-26 15:03:45:283 INFO  OServerConfigAuthenticator is active [OServerConfigAuthenticator]
2016-10-26 15:03:45:285 INFO  OSystemUserAuthenticator is active [OSystemUserAuthenticator]
2016-10-26 15:03:45:337 INFO  Installed GREMLIN language v.2.6.0 - graph.pool.max=50 [OGraphServerHandler]
2016-10-26 15:03:45:657 WARNI {db=OSystem} Storage 'OSystem' was not closed properly. Will try to recover from write ahead log [OLocalPaginatedStorage]
2016-10-26 15:03:45:659 INFO  {db=OSystem} Looking for last checkpoint... [OLocalPaginatedStorage]
2016-10-26 15:03:45:942 INFO  {db=OSystem} FULL checkpoint found. [OLocalPaginatedStorage]
2016-10-26 15:03:45:944 INFO  {db=OSystem} Data restore procedure from full checkpoint is started. Restore is performed from LSN OLogSequenceNumber{segment=0, position=623350} [OLocalPaginatedStorage]
2016-10-26 15:03:45:949 WARNI {db=OSystem} Record com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OCheckpointEndRecord{lsn=OLogSequenceNumber{segment=0, position=623374}} will be skipped during data restore [OLocalPaginatedStorage]
2016-10-26 15:03:45:950 INFO  {db=OSystem} 1 operations were processed, current LSN is OLogSequenceNumber{segment=0, position=623374} last LSN is OLogSequenceNumber{segment=0, position=623380} [OLocalPaginatedStorage]
2016-10-26 15:03:45:951 INFO  {db=OSystem} Storage data recover was completed [OLocalPaginatedStorage]com.orientechnologies.orient.core.exception.OConfigurationException: Error on deleting 'replicator' user
    at com.orientechnologies.orient.server.distributed.impl.ODistributedAbstractPlugin.config(ODistributedAbstractPlugin.java:173)
    at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.config(OHazelcastPlugin.java:98)
    at com.orientechnologies.orient.server.OServer.registerPlugins(OServer.java:1230)
    at com.orientechnologies.orient.server.OServer.activate(OServer.java:397)
    at com.orientechnologies.orient.server.OServerMain$1.run(OServerMain.java:46)
Caused by: java.io.FileNotFoundException: /home/orientdb/orientdb-community/config/orientdb-server-config.xml (Permission denied)
    at java.io.FileOutputStream.open0(Native Method)
    at java.io.FileOutputStream.open(FileOutputStream.java:270)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
    at java.io.FileWriter.<init>(FileWriter.java:90)
    at com.orientechnologies.orient.server.config.OServerConfigurationLoaderXml.save(OServerConfigurationLoaderXml.java:122)
    at com.orientechnologies.orient.server.config.OServerConfigurationManager.saveConfiguration(OServerConfigurationManager.java:107)
    at com.orientechnologies.orient.server.OServer.dropUser(OServer.java:786)
    at com.orientechnologies.orient.server.distributed.impl.ODistributedAbstractPlugin.config(ODistributedAbstractPlugin.java:171)
    ... 4 more

Please note:

  • It cannot write the pid file. The pid file location can be changed in ~/orientdb-community/bin/server.sh
./server.sh: 108: ./server.sh: cannot create /home/orientdb/orientdb-community/bin/orient.pid: Permission denied
  • We need to set the proper amount of Maximum Heap
2016-10-26 15:03:44:486 WARNI Not enough physical memory available for DISKCACHE: 2,938MB (heap=1,820MB direct=524,288MB). Set lower Maximum Heap (-Xmx setting on JVM) and restart OrientDB. Now running with DISKCACHE=256MB [orientechnologies]
  • The file /home/orientdb/orientdb-community/config/orientdb-server-config.xml need to be writable.
Caused by: java.io.FileNotFoundException: /home/orientdb/orientdb-community/config/orientdb-server-config.xml (Permission denied)

#7 Updated by Luca Frosini about 3 years ago

I just notice that root own orientdb directory (i.e. orientdb-community-2.2.12).
If there is no reason I ignore, I think is better to assign the ownership of orientdb directory to orientdb user because it runs as orientdb user.

#8 Updated by Luca Frosini about 3 years ago

@andrea.dellamico@isti.cnr.it fixed the ansible script. I run it. I'll check if everything is ok now.

#9 Updated by Luca Frosini about 3 years ago

  • % Done changed from 50 to 80

Seem that everything is configured in the proper way. Unfortunately hooks tag is missed in ~/orientdb-community/config/orientdb-server-config.xml

<hooks>
   <hook class="org.gcube.informationsystem.orientdb.hooks.HeaderHook" position="REGULAR"/>
</hooks>

#10 Updated by Luca Frosini about 3 years ago

The tags is created but at startime the xml is rewritten so that is not at the end as expected and I didn't notice it.

#11 Updated by Luca Frosini about 3 years ago

The last thing to be fixed in ansible script is that is a new version of hooks jas is available the new one is installed but the old one is not removed.

-rw-r--r--  1 root root 21006619 ott 26 12:15 resource-registry-orientdb-hooks-1.0.1-20161026.093908-29-jar-with-dependencies.jar
-rw-r--r--  1 root root  4253240 ott 26 18:14 resource-registry-orientdb-hooks-1.0.1-20161026.161036-30-jar-with-dependencies.jar

I'm going to remove the old one manually but it should be fixed in the ansible script.

#12 Updated by Andrea Dell'Amico about 3 years ago

Luca Frosini wrote:

The last thing to be fixed in ansible script is that is a new version of hooks jas is available the new one is installed but the old one is not removed.

-rw-r--r--  1 root root 21006619 ott 26 12:15 resource-registry-orientdb-hooks-1.0.1-20161026.093908-29-jar-with-dependencies.jar
-rw-r--r--  1 root root  4253240 ott 26 18:14 resource-registry-orientdb-hooks-1.0.1-20161026.161036-30-jar-with-dependencies.jar

I'm going to remove the old one manually but it should be fixed in the ansible script.

The only way I see is adding a task that removes a list of files. I added the task to the orientdb playbook, and the variable:

orientdb_hooks_to_be_removed:
  - resource-registry-orientdb-hooks-1.0.1-20161026.093908-29-jar-with-dependencies.jar  

#13 Updated by Luca Frosini about 3 years ago

I thought about this proposal but it is difficult to keep aligned.
Can we remove all files matching resource-registry-orientdb-hooks-*-jar-with-dependencies.jar before installing the new one.

#14 Updated by Andrea Dell'Amico about 3 years ago

Luca Frosini wrote:

I thought about this proposal but it is difficult to keep aligned.
Can we remove all files matching resource-registry-orientdb-hooks-*-jar-with-dependencies.jar before installing the new one.

In this way we are going to remove all the hook files at every playbook run, install the current one again, and then restart all the nodes. It seems more riskier that leaving an old file by mistake, to me.

#15 Updated by Luca Frosini about 3 years ago

This playbook is launched to upgrade orient or the hook. In both cases orientdb must be restarted to have effect.
Moreover if two hooks are present and orientdb is restarted we don't know which one is loaded.

#16 Updated by Andrea Dell'Amico about 3 years ago

  • % Done changed from 80 to 100
  • Status changed from In Progress to Feedback

OK, I adjusted the task that removes the old hooks.

#17 Updated by Luca Frosini about 3 years ago

  • Status changed from Feedback to Closed

thanks a lot

#18 Updated by Luca Frosini over 1 year ago

  • Related to Task #12389: Investigate a different configuration for orientdb cluster added

Also available in: Atom PDF