Showing detail for thumbnail request 2191833:576283
Logfile is 27573.5 MB, only reading last 1000.0 MB
png 128 x 74
https://thumbnails-earthtime.cmucreatelab.org/thumbnail?root=https%3A%2F%2Fcah.earthtime.org%2Fexplore%23v%3D-1.78111%2C0%2C-0.156%2ClatLng%26t%3D1.54%26ps%3D0%26l%3Dblsat%26bt%3D19840101%26et%3D19840101%26startDwell%3D0%26endDwell%3D0%26fps%3D10&width=128&height=74&format=png&fps=10&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot
Fatal error
Thumbnail delegation to hal40-thumbnailworker.createlab.org failed with status code 400 and response body 400: Bad Request<br><html><body><pre>Oh gosh, failed many times to initialize chrome, aborting
/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:155:in `make_chrome'
/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:283:in `start_thumbnail_from_screenshot'
/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:858:in `compute_thumbnail'
/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1240:in `serve_thumbnail'
/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1340:in `<main>'</pre><hr><pre>{
"SCRIPT_URL": "/thumbnail",
"SCRIPT_URI": "https://hal40-thumbnailworker.createlab.org/thumbnail",
"HTTPS": "on",
"SSL_TLS_SNI": "hal40-thumbnailworker.createlab.o
Last checkpoint
2026-05-26 17:39:17.891 THUMB 2191833:576283:0 CHECKPOINTTHUMBNAIL FATALERROR {"REMOTE_ADDR":"128.2.218.84","HTTP_USER_AGENT":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/127.0.6533.119 Safari/537.36","HTTP_REFERER":"https://cah.earthtime.org/","delegatedTo":"https://hal40-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fcah.earthtime.org%2Fexplore%23v%3D-1.78111%2C0%2C-0.156%2ClatLng%26t%3D1.54%26ps%3D0%26l%3Dblsat%26bt%3D19840101%26et%3D19840
Last line
2026-05-26 17:39:17.909 THUMB 2191833:576283:0 request.body {"service":"Thumbnails","datetime":"2026-05-26T17:39:17-04:00","host":"-","level":"up","summary":"Last thumbnail failed","details":null,"payload":{},"valid_for_secs":null}
Referer: https://cah.earthtime.org/ IP: 128.2.218.84 Browser: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/127.0.6533.119 Safari/537.36
Root
https://cah.earthtime.org/explore#v=-1.78111,0,-0.156,latLng&t=1.54&ps=0&l=blsat&bt=19840101&et=19840101&startDwell=0&endDwell=0&fps=10
Delegated to
https://hal40-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fcah.earthtime.org%2Fexplore%23v%3D-1.78111%2C0%2C-0.156%2ClatLng%26t%3D1.54%26ps%3D0%26l%3Dblsat%26bt%3D19840101%26et%3D19840101%26startDwell%3D0%26endDwell%3D0%26fps%3D10&width=128&height=74&format=png&fps=10&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&workerepoch=1779831040476
Raw stats
{"REMOTE_ADDR": "128.2.218.84", "HTTP_USER_AGENT": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/127.0.6533.119 Safari/537.36", "HTTP_REFERER": "https://cah.earthtime.org/", "delegatedTo": "https://hal40-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fcah.earthtime.org%2Fexplore%23v%3D-1.78111%2C0%2C-0.156%2ClatLng%26t%3D1.54%26ps%3D0%26l%3Dblsat%26bt%3D19840101%26et%3D19840101%26startDwell%3D0%26endDwell%3D0%26fps%3D10&width=128&height=74&format=png&fps=10&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&workerepoch=1779831040476", "FATALERROR": "Thumbnail delegation to hal40-thumbnailworker.createlab.org failed with status code 400 and response body 400: Bad Request<br><html><body><pre>Oh gosh, failed many times to initialize chrome, aborting\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:155:in `make_chrome'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:283:in `start_thumbnail_from_screenshot'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:858:in `compute_thumbnail'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1240:in `serve_thumbnail'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1340:in `<main>'</pre><hr><pre>{\n \"SCRIPT_URL\": \"/thumbnail\",\n \"SCRIPT_URI\": \"https://hal40-thumbnailworker.createlab.org/thumbnail\",\n \"HTTPS\": \"on\",\n \"SSL_TLS_SNI\": \"hal40-thumbnailworker.createlab.o"}
More: grep 2191833:576283 /t/thumbnails-v2.createlab.org/log.txt
Full log
2026-05-26 17:27:15.412 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:16.412 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:17.412 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:18.413 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:19.413 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:20.413 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:21.414 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:22.414 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:23.414 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:24.414 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:25.415 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:26.415 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:27.415 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:28.416 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:29.416 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:30.416 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:31.416 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:32.417 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:33.417 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:34.417 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:35.418 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:36.418 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:37.418 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:38.418 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:39.419 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:40.419 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:41.419 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:42.420 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:43.420 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:44.420 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:45.420 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:46.421 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:47.421 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:48.422 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:49.422 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:50.423 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:51.423 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:52.423 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:53.424 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:54.424 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:55.424 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:56.424 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:57.425 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:58.425 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:27:59.425 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:00.426 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:01.426 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:02.426 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:03.427 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:04.427 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:05.427 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:06.427 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:07.428 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:08.428 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:09.428 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:10.429 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:11.429 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:12.429 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:13.429 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:14.430 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:15.430 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:16.430 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:17.431 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:18.431 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:19.431 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:20.431 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:21.432 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:22.432 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:23.432 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:24.433 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:25.433 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:26.433 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:27.433 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:28.434 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:29.434 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:30.434 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:31.435 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:32.435 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:33.435 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:34.436 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:35.436 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:36.436 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:37.436 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:38.437 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:39.437 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:40.437 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:41.438 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:42.438 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:43.438 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:44.439 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:45.439 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:46.439 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:47.439 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:48.440 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:49.440 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:50.440 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:51.441 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:52.441 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:53.441 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:54.442 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:55.442 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:56.442 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:57.443 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:58.443 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:28:59.443 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:00.444 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:01.444 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:02.444 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:03.444 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:04.445 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:05.445 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:06.445 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:07.446 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:08.446 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:09.446 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:10.447 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:11.447 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:12.447 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:13.447 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:14.448 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:15.448 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:16.448 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:17.449 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:18.449 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:19.449 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:20.450 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:21.450 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:22.450 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:23.450 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:24.452 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:25.452 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:26.452 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:27.452 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:28.453 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:29.453 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:30.453 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:31.454 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:32.454 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:33.454 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:34.455 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:35.455 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:36.455 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:37.456 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:38.456 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:39.456 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:40.456 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:41.457 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:42.457 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:43.457 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:44.458 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:45.458 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:46.458 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:47.459 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:48.459 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:49.459 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:50.460 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:51.460 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:52.460 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:53.460 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:54.461 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:55.461 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:56.461 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:57.462 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:58.462 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:29:59.462 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:00.462 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:01.463 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:02.463 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:03.463 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:04.464 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:05.464 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:06.464 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:07.465 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:08.465 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:09.465 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:10.466 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:11.466 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:12.466 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:13.466 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:14.467 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:15.467 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:16.467 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:17.468 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:18.468 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:19.468 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:20.469 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:21.469 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:22.469 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:23.469 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:24.470 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:25.470 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:26.470 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:27.471 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:28.471 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:29.471 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:30.472 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:31.472 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:32.472 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:33.472 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:34.473 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:35.473 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:36.473 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:37.474 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:38.474 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:39.474 THUMB 2191833:576283:0 Cannot lock compute lockfile; waiting for another process to finish computing
2026-05-26 17:30:40.475 THUMB 2191833:576283:0 Not found in cache; computing
2026-05-26 17:30:40.475 THUMB 2191833:576283:0 STARTTHUMBNAIL https://thumbnails-earthtime.cmucreatelab.org/thumbnail?root=https%3A%2F%2Fcah.earthtime.org%2Fexplore%23v%3D-1.78111%2C0%2C-0.156%2ClatLng%26t%3D1.54%26ps%3D0%26l%3Dblsat%26bt%3D19840101%26et%3D19840101%26startDwell%3D0%26endDwell%3D0%26fps%3D10&width=128&height=74&format=png&fps=10&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot
2026-05-26 17:30:40.475 THUMB 2191833:576283:0 CHECKPOINTTHUMBNAIL START {"REMOTE_ADDR":"128.2.218.84","HTTP_USER_AGENT":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/127.0.6533.119 Safari/537.36","HTTP_REFERER":"https://cah.earthtime.org/"}
2026-05-26 17:30:40.475 THUMB 2191833:576283:0 Captured resource lock /t/thumbnails-v2.createlab.org/locks/0049+hal40-thumbnailworker.createlab.org from /t/thumbnails-v2.createlab.org/locks
2026-05-26 17:30:40.475 THUMB 2191833:576283:0 Thumbnail worker hal40-thumbnailworker.createlab.org
2026-05-26 17:30:40.475 THUMB 2191833:576283:0 CHECKPOINTTHUMBNAIL DELEGATION_START {"REMOTE_ADDR":"128.2.218.84","HTTP_USER_AGENT":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/127.0.6533.119 Safari/537.36","HTTP_REFERER":"https://cah.earthtime.org/","delegatedTo":"https://hal40-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fcah.earthtime.org%2Fexplore%23v%3D-1.78111%2C0%2C-0.156%2ClatLng%26t%3D1.54%26ps%3D0%26l%3Dblsat%26bt%3D19840101%26et%3D19840101%26startDwell%3D0%26endDwell%3D0%26fps%3D10&width=128&height=74&format=png&fps=10&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&workerepoch=1779831040476"}
2026-05-26 17:30:40.475 THUMB 2191833:576283:0 Delegating thumbnail to hal40-thumbnailworker.createlab.org
2026-05-26 17:39:17.890 THUMB 2191833:576283:0 Thumbnail returned from hal40-thumbnailworker.createlab.org, status code 400, body length 3358
2026-05-26 17:39:17.891 THUMB 2191833:576283:0 CHECKPOINTTHUMBNAIL DELEGATION_FINISHED {"REMOTE_ADDR":"128.2.218.84","HTTP_USER_AGENT":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/127.0.6533.119 Safari/537.36","HTTP_REFERER":"https://cah.earthtime.org/","delegatedTo":"https://hal40-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fcah.earthtime.org%2Fexplore%23v%3D-1.78111%2C0%2C-0.156%2ClatLng%26t%3D1.54%26ps%3D0%26l%3Dblsat%26bt%3D19840101%26et%3D19840101%26startDwell%3D0%26endDwell%3D0%26fps%3D10&width=128&height=74&format=png&fps=10&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&workerepoch=1779831040476"}
2026-05-26 17:39:17.891 THUMB 2191833:576283:0 CHECKPOINTTHUMBNAIL FATALERROR {"REMOTE_ADDR":"128.2.218.84","HTTP_USER_AGENT":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/127.0.6533.119 Safari/537.36","HTTP_REFERER":"https://cah.earthtime.org/","delegatedTo":"https://hal40-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fcah.earthtime.org%2Fexplore%23v%3D-1.78111%2C0%2C-0.156%2ClatLng%26t%3D1.54%26ps%3D0%26l%3Dblsat%26bt%3D19840101%26et%3D19840101%26startDwell%3D0%26endDwell%3D0%26fps%3D10&width=128&height=74&format=png&fps=10&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&workerepoch=1779831040476","FATALERROR":"Thumbnail delegation to hal40-thumbnailworker.createlab.org failed with status code 400 and response body 400: Bad Request<br><html><body><pre>Oh gosh, failed many times to initialize chrome, aborting\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:155:in `make_chrome'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:283:in `start_thumbnail_from_screenshot'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:858:in `compute_thumbnail'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1240:in `serve_thumbnail'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1340:in `<main>'</pre><hr><pre>{\n \"SCRIPT_URL\": \"/thumbnail\",\n \"SCRIPT_URI\": \"https://hal40-thumbnailworker.createlab.org/thumbnail\",\n \"HTTPS\": \"on\",\n \"SSL_TLS_SNI\": \"hal40-thumbnailworker.createlab.org\",\n \"HTTP_ACCEPT_ENCODING\": \"gzip;q=1.0,deflate;q=0.6,identity;q=0.3\",\n \"HTTP_ACCEPT\": \"*/*\",\n \"HTTP_USER_AGENT\": \"Ruby\",\n \"HTTP_HOST\": \"hal40-thumbnailworker.createlab.org\",\n \"HTTP_CONNECTION\": \"close\",\n \"PATH\": \"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin\",\n \"SERVER_SIGNATURE\": \"<address>Apache/2.4.41 (Ubuntu) Server at hal40-thumbnailworker.createlab.org Port 443</address>\\n\",\n \"SERVER_SOFTWARE\": \"Apache/2.4.41 (Ubuntu)\",\n \"SERVER_NAME\": \"hal40-thumbnailworker.createlab.org\",\n \"SERVER_ADDR\": \"128.2.218.84\",\n \"SERVER_PORT\": \"443\",\n \"REMOTE_ADDR\": \"128.2.220.212\",\n \"DOCUMENT_ROOT\": \"/thumbnailworker/thumbnailworker.createlab.org/cache\",\n \"REQUEST_SCHEME\": \"https\",\n \"CONTEXT_PREFIX\": \"\",\n \"CONTEXT_DOCUMENT_ROOT\": \"/thumbnailworker/thumbnailworker.createlab.org/cache\",\n \"SERVER_ADMIN\": \"[no address given]\",\n \"SCRIPT_FILENAME\": \"/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb\",\n \"REMOTE_PORT\": \"46970\",\n \"GATEWAY_INTERFACE\": \"CGI/1.1\",\n \"SERVER_PROTOCOL\": \"HTTP/1.1\",\n \"REQUEST_METHOD\": \"GET\",\n \"QUERY_STRING\": \"cachepath=/thumbnail/root=https%3A%2F%2Fcah02Eearthtime02Eorg%2Fexplore023v03D-102E7811102C002C-002E1/5602ClatLng026t03D102E54026ps03D0026l03Dblsat026bt03D19840101026et03D19840101026/startDwell03D0026endDwell03D0026fps03D10026width=128026height=74026format=png026/fps=10026tileFormat=mp4026startDwell=0026endDwell=0026fromScreenshot026workerepo/ch=1779831040476\",\n \"REQUEST_URI\": \"/thumbnail?root=https%3A%2F%2Fcah.earthtime.org%2Fexplore%23v%3D-1.78111%2C0%2C-0.156%2ClatLng%26t%3D1.54%26ps%3D0%26l%3Dblsat%26bt%3D19840101%26et%3D19840101%26startDwell%3D0%26endDwell%3D0%26fps%3D10&width=128&height=74&format=png&fps=10&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&workerepoch=1779831040476\",\n \"SCRIPT_NAME\": \"/thumbnail\"\n}</pre><hr><pre>{\n \"width\": [\n \"128\"\n ],\n \"height\": [\n \"74\"\n ],\n \"format\": [\n \"png\"\n ],\n \"fps\": [\n \"10\"\n ],\n \"tileFormat\": [\n \"mp4\"\n ],\n \"startDwell\": [\n \"0\"\n ],\n \"endDwell\": [\n \"0\"\n ],\n \"fromScreenshot\": [\n\n ],\n \"workerepoch\": [\n \"1779831040476\"\n ],\n \"root\": [\n \"https://cah.earthtime.org/explore#v=-1.78111,0,-0.156,latLng&t=1.54&ps=0&l=blsat&bt=19840101&et=19840101&startDwell=0&endDwell=0&fps=10\"\n ]\n}</pre><hr>root: https://cah.earthtime.org/explore#v=-1.78111,0,-0.156,latLng&t=1.54&ps=0&l=blsat&bt=19840101&et=19840101&startDwell=0&endDwell=0&fps=10<br>\n/t/thumbnails-v2.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1095:in `delegate_thumbnail'\n/t/thumbnails-v2.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1235:in `serve_thumbnail'\n/t/thumbnails-v2.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1360:in `<main>'"}
2026-05-26 17:39:17.891 THUMB 2191833:576283:0 url is https://stat.createlab.org/api/log host stat.createlab.org port 443
2026-05-26 17:39:17.891 THUMB 2191833:576283:0 request.body {"service":"Thumbnails","datetime":"2026-05-26T17:39:17-04:00","host":"-","level":"info","summary":"Thumbnail failed 2191833:576283 on thumbnails-earthtime.cmucreatelab.org in 722.5 seconds","details":"<a href=\"https://hal40-thumbnailworker.createlab.org/status?id=2191833:576283\">More info</a>","payload":{},"valid_for_secs":null}
2026-05-26 17:39:17.909 THUMB 2191833:576283:0 url is https://stat.createlab.org/api/log host stat.createlab.org port 443
2026-05-26 17:39:17.909 THUMB 2191833:576283:0 request.body {"service":"Thumbnails","datetime":"2026-05-26T17:39:17-04:00","host":"-","level":"up","summary":"Last thumbnail failed","details":null,"payload":{},"valid_for_secs":null}