Showing detail for thumbnail request 536164:359064
Logfile is 27156.4 MB, only reading last 1000.0 MB
mp4 1280 x 720
https://thumbnails-v2.createlab.org/thumbnail?root=https%3A%2F%2Fbreathecam.org%23v%3D290%2C670%2C5247%2C3458%2Cpts%26t%3D970.19%26ps%3D50%26bt%3D20251103144354%26et%3D20251103144748%26startDwell%3D0%26endDwell%3D0%26d%3D2025-11-03%26s%3Daccan2%26fps%3D6&width=1280&height=720&format=mp4&fps=6&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&minimalUI
Fatal error
Thumbnail delegation to hal20-thumbnailworker.createlab.org failed with status code 400 and response body 400: Bad Request<br><html><body><pre>Oh gosh, failed many times to initialize chrome, aborting
/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:155:in `make_chrome'
/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:283:in `start_thumbnail_from_screenshot'
/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:858:in `compute_thumbnail'
/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1240:in `serve_thumbnail'
/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1340:in `<main>'</pre><hr><pre>{
"SCRIPT_URL": "/thumbnail",
"SCRIPT_URI": "https://hal20-thumbnailworker.createlab.org/thumbnail",
"HTTPS": "on",
"SSL_TLS_SNI": "hal20-thumbnailworker.createlab.o
Last checkpoint
2025-11-05 11:37:21.364 THUMB 536164:359064:0 CHECKPOINTTHUMBNAIL FATALERROR {"REMOTE_ADDR":"73.79.199.15","HTTP_USER_AGENT":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36","delegatedTo":"https://hal20-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fbreathecam.org%23v%3D290%2C670%2C5247%2C3458%2Cpts%26t%3D970.19%26ps%3D50%26bt%3D20251103144354%26et%3D20251103144748%26startDwell%3D0%26endDwell%3D0%26d%3D2025-11-03%
Last line
2025-11-05 11:37:21.387 THUMB 536164:359064:0 request.body {"service":"Thumbnails","datetime":"2025-11-05T11:37:21-05:00","host":"-","level":"up","summary":"Last thumbnail failed","details":null,"payload":{},"valid_for_secs":null}
IP: 73.79.199.15 Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36
Root
https://breathecam.org#v=290,670,5247,3458,pts&t=970.19&ps=50&bt=20251103144354&et=20251103144748&startDwell=0&endDwell=0&d=2025-11-03&s=accan2&fps=6
Delegated to
https://hal20-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fbreathecam.org%23v%3D290%2C670%2C5247%2C3458%2Cpts%26t%3D970.19%26ps%3D50%26bt%3D20251103144354%26et%3D20251103144748%26startDwell%3D0%26endDwell%3D0%26d%3D2025-11-03%26s%3Daccan2%26fps%3D6&width=1280&height=720&format=mp4&fps=6&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&minimalUI&workerepoch=1762359891723
Raw stats
{"REMOTE_ADDR": "73.79.199.15", "HTTP_USER_AGENT": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36", "delegatedTo": "https://hal20-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fbreathecam.org%23v%3D290%2C670%2C5247%2C3458%2Cpts%26t%3D970.19%26ps%3D50%26bt%3D20251103144354%26et%3D20251103144748%26startDwell%3D0%26endDwell%3D0%26d%3D2025-11-03%26s%3Daccan2%26fps%3D6&width=1280&height=720&format=mp4&fps=6&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&minimalUI&workerepoch=1762359891723", "FATALERROR": "Thumbnail delegation to hal20-thumbnailworker.createlab.org failed with status code 400 and response body 400: Bad Request<br><html><body><pre>Oh gosh, failed many times to initialize chrome, aborting\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:155:in `make_chrome'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:283:in `start_thumbnail_from_screenshot'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:858:in `compute_thumbnail'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1240:in `serve_thumbnail'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1340:in `<main>'</pre><hr><pre>{\n \"SCRIPT_URL\": \"/thumbnail\",\n \"SCRIPT_URI\": \"https://hal20-thumbnailworker.createlab.org/thumbnail\",\n \"HTTPS\": \"on\",\n \"SSL_TLS_SNI\": \"hal20-thumbnailworker.createlab.o"}
More: grep 536164:359064 /t/thumbnails-v2.createlab.org/log.txt
Full log
2025-11-05 11:21:01.650 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:02.650 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:03.650 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:04.650 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:05.651 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:06.651 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:07.651 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:08.652 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:09.652 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:10.652 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:11.653 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:12.653 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:13.653 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:14.654 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:15.654 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:16.654 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:17.654 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:18.655 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:19.655 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:20.655 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:21.656 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:22.656 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:23.656 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:24.657 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:25.657 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:26.657 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:27.658 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:28.658 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:29.658 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:30.658 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:31.659 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:32.659 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:33.659 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:34.660 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:35.660 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:36.660 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:37.661 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:38.661 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:39.661 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:40.662 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:41.662 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:42.662 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:43.662 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:44.663 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:45.663 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:46.663 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:47.664 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:48.664 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:49.664 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:50.665 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:51.665 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:52.665 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:53.665 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:54.666 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:55.666 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:56.666 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:57.667 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:58.667 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:21:59.667 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:00.668 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:01.668 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:02.668 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:03.669 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:04.669 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:05.669 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:06.670 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:07.670 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:08.670 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:09.671 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:10.671 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:11.671 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:12.672 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:13.672 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:14.672 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:15.672 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:16.673 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:17.673 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:18.673 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:19.674 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:20.674 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:21.674 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:22.675 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:23.675 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:24.675 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:25.675 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:26.676 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:27.676 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:28.676 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:29.677 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:30.677 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:31.677 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:32.678 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:33.678 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:34.678 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:35.679 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:36.679 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:37.679 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:38.679 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:39.680 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:40.680 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:41.680 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:42.681 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:43.681 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:44.681 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:45.682 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:46.682 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:47.682 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:48.682 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:49.683 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:50.683 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:51.683 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:52.684 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:53.684 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:54.684 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:55.685 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:56.685 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:57.685 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:58.686 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:22:59.686 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:00.686 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:01.687 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:02.687 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:03.687 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:04.687 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:05.688 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:06.688 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:07.688 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:08.689 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:09.689 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:10.689 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:11.690 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:12.690 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:13.690 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:14.690 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:15.691 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:16.692 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:17.692 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:18.693 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:19.693 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:20.693 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:21.694 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:22.694 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:23.694 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:24.695 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:25.695 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:26.695 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:27.695 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:28.696 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:29.696 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:30.696 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:31.697 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:32.697 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:33.697 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:34.698 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:35.698 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:36.698 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:37.699 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:38.699 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:39.700 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:40.700 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:41.700 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:42.701 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:43.701 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:44.701 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:45.702 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:46.702 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:47.702 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:48.703 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:49.703 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:50.703 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:51.703 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:52.704 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:53.704 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:54.704 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:55.705 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:56.705 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:57.705 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:58.706 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:23:59.706 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:00.706 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:01.707 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:02.707 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:03.707 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:04.707 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:05.708 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:06.708 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:07.708 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:08.709 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:09.709 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:10.709 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:11.710 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:12.710 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:13.710 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:14.711 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:15.711 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:16.711 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:17.712 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:18.712 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:19.712 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:20.712 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:21.713 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:22.713 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:23.713 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:24.714 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:25.714 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:26.714 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:27.715 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:28.715 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:29.715 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:30.716 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:31.716 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:32.716 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:33.716 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:34.717 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:35.717 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:36.717 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:37.718 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:38.718 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:39.718 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:40.719 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:41.719 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:42.719 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:43.719 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:44.720 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:45.720 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:46.720 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:47.721 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:48.721 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:49.721 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:50.722 THUMB 536164:359064:0 Cannot lock compute lockfile; waiting for another process to finish computing
2025-11-05 11:24:51.722 THUMB 536164:359064:0 Not found in cache; computing
2025-11-05 11:24:51.722 THUMB 536164:359064:0 STARTTHUMBNAIL https://thumbnails-v2.createlab.org/thumbnail?root=https%3A%2F%2Fbreathecam.org%23v%3D290%2C670%2C5247%2C3458%2Cpts%26t%3D970.19%26ps%3D50%26bt%3D20251103144354%26et%3D20251103144748%26startDwell%3D0%26endDwell%3D0%26d%3D2025-11-03%26s%3Daccan2%26fps%3D6&width=1280&height=720&format=mp4&fps=6&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&minimalUI
2025-11-05 11:24:51.722 THUMB 536164:359064:0 CHECKPOINTTHUMBNAIL START {"REMOTE_ADDR":"73.79.199.15","HTTP_USER_AGENT":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36"}
2025-11-05 11:24:51.722 THUMB 536164:359064:0 Captured resource lock /t/thumbnails-v2.createlab.org/locks/0053+hal20-thumbnailworker.createlab.org from /t/thumbnails-v2.createlab.org/locks
2025-11-05 11:24:51.722 THUMB 536164:359064:0 Thumbnail worker hal20-thumbnailworker.createlab.org
2025-11-05 11:24:51.723 THUMB 536164:359064:0 CHECKPOINTTHUMBNAIL DELEGATION_START {"REMOTE_ADDR":"73.79.199.15","HTTP_USER_AGENT":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36","delegatedTo":"https://hal20-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fbreathecam.org%23v%3D290%2C670%2C5247%2C3458%2Cpts%26t%3D970.19%26ps%3D50%26bt%3D20251103144354%26et%3D20251103144748%26startDwell%3D0%26endDwell%3D0%26d%3D2025-11-03%26s%3Daccan2%26fps%3D6&width=1280&height=720&format=mp4&fps=6&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&minimalUI&workerepoch=1762359891723"}
2025-11-05 11:24:51.723 THUMB 536164:359064:0 Delegating thumbnail to hal20-thumbnailworker.createlab.org
2025-11-05 11:37:21.364 THUMB 536164:359064:0 Thumbnail returned from hal20-thumbnailworker.createlab.org, status code 400, body length 3466
2025-11-05 11:37:21.364 THUMB 536164:359064:0 CHECKPOINTTHUMBNAIL DELEGATION_FINISHED {"REMOTE_ADDR":"73.79.199.15","HTTP_USER_AGENT":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36","delegatedTo":"https://hal20-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fbreathecam.org%23v%3D290%2C670%2C5247%2C3458%2Cpts%26t%3D970.19%26ps%3D50%26bt%3D20251103144354%26et%3D20251103144748%26startDwell%3D0%26endDwell%3D0%26d%3D2025-11-03%26s%3Daccan2%26fps%3D6&width=1280&height=720&format=mp4&fps=6&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&minimalUI&workerepoch=1762359891723"}
2025-11-05 11:37:21.364 THUMB 536164:359064:0 CHECKPOINTTHUMBNAIL FATALERROR {"REMOTE_ADDR":"73.79.199.15","HTTP_USER_AGENT":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36","delegatedTo":"https://hal20-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fbreathecam.org%23v%3D290%2C670%2C5247%2C3458%2Cpts%26t%3D970.19%26ps%3D50%26bt%3D20251103144354%26et%3D20251103144748%26startDwell%3D0%26endDwell%3D0%26d%3D2025-11-03%26s%3Daccan2%26fps%3D6&width=1280&height=720&format=mp4&fps=6&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&minimalUI&workerepoch=1762359891723","FATALERROR":"Thumbnail delegation to hal20-thumbnailworker.createlab.org failed with status code 400 and response body 400: Bad Request<br><html><body><pre>Oh gosh, failed many times to initialize chrome, aborting\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:155:in `make_chrome'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:283:in `start_thumbnail_from_screenshot'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:858:in `compute_thumbnail'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1240:in `serve_thumbnail'\n/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1340:in `<main>'</pre><hr><pre>{\n \"SCRIPT_URL\": \"/thumbnail\",\n \"SCRIPT_URI\": \"https://hal20-thumbnailworker.createlab.org/thumbnail\",\n \"HTTPS\": \"on\",\n \"SSL_TLS_SNI\": \"hal20-thumbnailworker.createlab.org\",\n \"HTTP_ACCEPT_ENCODING\": \"gzip;q=1.0,deflate;q=0.6,identity;q=0.3\",\n \"HTTP_ACCEPT\": \"*/*\",\n \"HTTP_USER_AGENT\": \"Ruby\",\n \"HTTP_HOST\": \"hal20-thumbnailworker.createlab.org\",\n \"HTTP_CONNECTION\": \"close\",\n \"PATH\": \"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin\",\n \"SERVER_SIGNATURE\": \"<address>Apache/2.4.41 (Ubuntu) Server at hal20-thumbnailworker.createlab.org Port 443</address>\\n\",\n \"SERVER_SOFTWARE\": \"Apache/2.4.41 (Ubuntu)\",\n \"SERVER_NAME\": \"hal20-thumbnailworker.createlab.org\",\n \"SERVER_ADDR\": \"128.237.154.91\",\n \"SERVER_PORT\": \"443\",\n \"REMOTE_ADDR\": \"128.2.220.212\",\n \"DOCUMENT_ROOT\": \"/thumbnailworker/thumbnailworker.createlab.org/cache\",\n \"REQUEST_SCHEME\": \"https\",\n \"CONTEXT_PREFIX\": \"\",\n \"CONTEXT_DOCUMENT_ROOT\": \"/thumbnailworker/thumbnailworker.createlab.org/cache\",\n \"SERVER_ADMIN\": \"[no address given]\",\n \"SCRIPT_FILENAME\": \"/thumbnailworker/thumbnailworker.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb\",\n \"REMOTE_PORT\": \"39762\",\n \"GATEWAY_INTERFACE\": \"CGI/1.1\",\n \"SERVER_PROTOCOL\": \"HTTP/1.1\",\n \"REQUEST_METHOD\": \"GET\",\n \"QUERY_STRING\": \"cachepath=/thumbnail/root=https%3A%2F%2Fbreathecam02Eorg023v03D29002C67002C524702C345802Cpts026t03D97/002E19026ps03D50026bt03D20251103144354026et03D20251103144748026startDwell03D0026/endDwell03D0026d03D2025-11-03026s03Daccan2026fps03D6026width=1280026height=72002/6format=mp4026fps=6026tileFormat=mp4026startDwell=0026endDwell=0026fromScreensho/t026minimalUI026workerepoch=1762359891723\",\n \"REQUEST_URI\": \"/thumbnail?root=https%3A%2F%2Fbreathecam.org%23v%3D290%2C670%2C5247%2C3458%2Cpts%26t%3D970.19%26ps%3D50%26bt%3D20251103144354%26et%3D20251103144748%26startDwell%3D0%26endDwell%3D0%26d%3D2025-11-03%26s%3Daccan2%26fps%3D6&width=1280&height=720&format=mp4&fps=6&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&minimalUI&workerepoch=1762359891723\",\n \"SCRIPT_NAME\": \"/thumbnail\"\n}</pre><hr><pre>{\n \"width\": [\n \"1280\"\n ],\n \"height\": [\n \"720\"\n ],\n \"format\": [\n \"mp4\"\n ],\n \"fps\": [\n \"6\"\n ],\n \"tileFormat\": [\n \"mp4\"\n ],\n \"startDwell\": [\n \"0\"\n ],\n \"endDwell\": [\n \"0\"\n ],\n \"fromScreenshot\": [\n\n ],\n \"minimalUI\": [\n\n ],\n \"workerepoch\": [\n \"1762359891723\"\n ],\n \"root\": [\n \"https://breathecam.org#v=290,670,5247,3458,pts&t=970.19&ps=50&bt=20251103144354&et=20251103144748&startDwell=0&endDwell=0&d=2025-11-03&s=accan2&fps=6\"\n ]\n}</pre><hr>root: https://breathecam.org#v=290,670,5247,3458,pts&t=970.19&ps=50&bt=20251103144354&et=20251103144748&startDwell=0&endDwell=0&d=2025-11-03&s=accan2&fps=6<br>\n/t/thumbnails-v2.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1095:in `delegate_thumbnail'\n/t/thumbnails-v2.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1235:in `serve_thumbnail'\n/t/thumbnails-v2.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1360:in `<main>'"}
2025-11-05 11:37:21.365 THUMB 536164:359064:0 url is https://stat.createlab.org/api/log host stat.createlab.org port 443
2025-11-05 11:37:21.365 THUMB 536164:359064:0 request.body {"service":"Thumbnails","datetime":"2025-11-05T11:37:21-05:00","host":"-","level":"info","summary":"Thumbnail failed 536164:359064 on thumbnails-v2.createlab.org in 979.7 seconds","details":"<a href=\"https://hal20-thumbnailworker.createlab.org/status?id=536164:359064\">More info</a>","payload":{},"valid_for_secs":null}
2025-11-05 11:37:21.387 THUMB 536164:359064:0 url is https://stat.createlab.org/api/log host stat.createlab.org port 443
2025-11-05 11:37:21.387 THUMB 536164:359064:0 request.body {"service":"Thumbnails","datetime":"2025-11-05T11:37:21-05:00","host":"-","level":"up","summary":"Last thumbnail failed","details":null,"payload":{},"valid_for_secs":null}