Skip to content

JdbcSQLException: Timeout trying to lock table "METRIC" when visiting /compactions #37

@lins05

Description

@lins05

HBase version: 0.94.15

Hannibal has been running for several months. Today when I try to open the compactions page, the server returns an error page (The other pages like tables/regions are good).The logs/application.log file shows:

2015-01-31 03:31:14,741 - [ERROR] - from application in play-akka.actor.actions-dispatcher-1167 


! @6ieie9fjm - Internal server error, for request [GET /compactions] ->

play.core.ActionInvoker$$anonfun$receive$1$$anon$1: Execution exception [[JdbcSQLException: Timeout trying to lock table "METRIC"; SQL statement:
SELECT       id, target, name, last_value, last_update, target_desc     FROM       metric     WHERE       name=? [50200-158]]]
    at play.core.ActionInvoker$$anonfun$receive$1.apply(Invoker.scala:134) [play_2.9.1-2.0.4.jar:2.0.4]
    at play.core.ActionInvoker$$anonfun$receive$1.apply(Invoker.scala:115) [play_2.9.1-2.0.4.jar:2.0.4]
    at akka.actor.Actor$class.apply(Actor.scala:318) [akka-actor-2.0.2.jar:2.0.2]
    at play.core.ActionInvoker.apply(Invoker.scala:113) [play_2.9.1-2.0.4.jar:2.0.4]
    at akka.actor.ActorCell.invoke(ActorCell.scala:626) [akka-actor-2.0.2.jar:2.0.2]
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:197) [akka-actor-2.0.2.jar:2.0.2]
    at akka.dispatch.Mailbox.run(Mailbox.scala:179) [akka-actor-2.0.2.jar:2.0.2]
    at akka.dispatch.ForkJoinExecutorConfigurator$MailboxExecutionTask.exec(AbstractDispatcher.scala:516) [akka-actor-2.0.2.jar:2.0.2]
    at akka.jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:259) [akka-actor-2.0.2.jar:2.0.2]
    at akka.jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:975) [akka-actor-2.0.2.jar:2.0.2]
    at akka.jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1479) [akka-actor-2.0.2.jar:2.0.2]
    at akka.jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) [akka-actor-2.0.2.jar:2.0.2]
Caused by: org.h2.jdbc.JdbcSQLException: Timeout trying to lock table "METRIC"; SQL statement:
SELECT       id, target, name, last_value, last_update, target_desc     FROM       metric     WHERE       name=? [50200-158]
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:329) ~[h2-1.3.158.jar:1.3.158]
    at org.h2.message.DbException.get(DbException.java:169) ~[h2-1.3.158.jar:1.3.158]
    at org.h2.message.DbException.get(DbException.java:146) ~[h2-1.3.158.jar:1.3.158]
    at org.h2.table.RegularTable.doLock(RegularTable.java:506) ~[h2-1.3.158.jar:1.3.158]
    at org.h2.table.RegularTable.lock(RegularTable.java:440) ~[h2-1.3.158.jar:1.3.158]
    at org.h2.table.TableFilter.lock(TableFilter.java:142) ~[h2-1.3.158.jar:1.3.158]
    at org.h2.command.dml.Select.queryWithoutCache(Select.java:590) ~[h2-1.3.158.jar:1.3.158]
    at org.h2.command.dml.Query.query(Query.java:298) ~[h2-1.3.158.jar:1.3.158]
    at org.h2.command.dml.Query.query(Query.java:268) ~[h2-1.3.158.jar:1.3.158]
    at org.h2.command.dml.Query.query(Query.java:37) ~[h2-1.3.158.jar:1.3.158]
    at org.h2.command.CommandContainer.query(CommandContainer.java:80) ~[h2-1.3.158.jar:1.3.158]
    at org.h2.command.Command.executeQuery(Command.java:181) ~[h2-1.3.158.jar:1.3.158]
    at org.h2.jdbc.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:96) ~[h2-1.3.158.jar:1.3.158]
    at com.jolbox.bonecp.PreparedStatementHandle.executeQuery(PreparedStatementHandle.java:172) ~[bonecp-0.7.1.RELEASE.jar:0.7.1.RELEASE]
    at anorm.Sql$class.resultSet(Anorm.scala:429) ~[anorm_2.9.1-2.0.4.jar:2.0.4]
    at anorm.SimpleSql.resultSet(Anorm.scala:359) ~[anorm_2.9.1-2.0.4.jar:2.0.4]
    at anorm.Sql$class.apply(Anorm.scala:427) ~[anorm_2.9.1-2.0.4.jar:2.0.4]
    at anorm.SimpleSql.apply(Anorm.scala:359) ~[anorm_2.9.1-2.0.4.jar:2.0.4]
    at models.MetricDef$$anonfun$findByName$1.apply(Metric.scala:73) ~[hannibal_2.9.1-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]
    at models.MetricDef$$anonfun$findByName$1.apply(Metric.scala:72) ~[hannibal_2.9.1-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]
    at play.api.db.DBApi$class.withConnection(DB.scala:79) ~[play_2.9.1-2.0.4.jar:2.0.4]
    at play.api.db.BoneCPApi.withConnection(DB.scala:273) ~[play_2.9.1-2.0.4.jar:2.0.4]
    at play.api.db.DB$$anonfun$withConnection$3.apply(DB.scala:158) ~[play_2.9.1-2.0.4.jar:2.0.4]
    at play.api.db.DB$$anonfun$withConnection$3.apply(DB.scala:158) ~[play_2.9.1-2.0.4.jar:2.0.4]
    at scala.Option.map(Option.scala:133) ~[scala-library.jar:na]
    at play.api.db.DB$.withConnection(DB.scala:158) ~[play_2.9.1-2.0.4.jar:2.0.4]
    at models.MetricDef$.findByName(Metric.scala:72) ~[hannibal_2.9.1-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]
    at utils.MetricUtil$.findLongestCompactionInLastWeek(MetricUtil.scala:39) ~[hannibal_2.9.1-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]
    at controllers.Compactions$$anonfun$index$1.apply(Compactions.scala:13) ~[hannibal_2.9.1-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]
    at controllers.Compactions$$anonfun$index$1.apply(Compactions.scala:12) ~[hannibal_2.9.1-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]
    at play.api.mvc.Action$$anon$1.apply(Action.scala:170) ~[play_2.9.1-2.0.4.jar:2.0.4]
    at play.core.ActionInvoker$$anonfun$receive$1$$anonfun$6.apply(Invoker.scala:126) ~[play_2.9.1-2.0.4.jar:2.0.4]
    at play.core.ActionInvoker$$anonfun$receive$1$$anonfun$6.apply(Invoker.scala:126) ~[play_2.9.1-2.0.4.jar:2.0.4]
    at play.utils.Threads$.withContextClassLoader(Threads.scala:17) ~[play_2.9.1-2.0.4.jar:2.0.4]
    at play.core.ActionInvoker$$anonfun$receive$1.apply(Invoker.scala:125) [play_2.9.1-2.0.4.jar:2.0.4]
    ... 11 common frames omitted

Here are the files in data/ foder:

$ ls -lh data/
total 118M
-rw-r--r-- 1 user group  40M Sep 20 14:38 metrics.h2.h2.db
-rw-r--r-- 1 user group  9 09:42 metrics.h2.lock.db
-rw-r--r-- 1 user group 18M Jan 31 03:42 metrics.h2.trace.db
-rw-r--r-- 1 user group 65M Dec 24 04:38 metrics.h2.trace.db.old

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions