Showing detail for thumbnail request 1715138:063161

Logfile is 27066.9 MB, only reading last 1000.0 MB

Failed thumbnail 1715138:063161 2025-08-20 07:44:56

zip 540 x 540 https://thumbnails-earthtime.cmucreatelab.org/thumbnail?root=https%3A%2F%2Fheadless.earthtime.org%2F%23l%3Dmb_labeled%2Csmell_my_city_pgh_reports_top2025%2Cplume_202508190400_202508200400%26bt%3D20250819040000%26et%3D20250819120000%26v%3D40.42532%2C-79.91643%2C9.233%2ClatLng%26ps%3D2400%26startDwell%3D0%26endDwell%3D0&&width=540&height=540&format=zip&fps=30&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&disableUI&redo=0

Fatal error

Thumbnail delegation to hal40-thumbnailworker.createlab.org failed with status code 500 and response body <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator at 
 [no address given] to inform them of the time this error occurred,
 and the actions you performed just before this error.</p>
<p>More information about this error may be available
in the server error log.</p>
<hr>
<address>Apache/2.4.41 (Ubuntu) Server at hal40-thumbnailworker.createlab.org Port 443</address>
</body></html>

/t/thumbnails-v2.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1095:in `delegate_thumbnail'
/t/thumbnails-v2.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1235:in `serve_thumbnail'
/t/thumbnails-v2.createlab.org/timemachine-th

Last checkpoint

2025-08-20 07:45:03.716 THUMB 1715138:063161:0 CHECKPOINTTHUMBNAIL FATALERROR {"REMOTE_ADDR":"128.2.218.94","HTTP_USER_AGENT":"Python-urllib/3.7","delegatedTo":"https://hal40-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fheadless.earthtime.org%2F%23l%3Dmb_labeled%2Csmell_my_city_pgh_reports_top2025%2Cplume_202508190400_202508200400%26bt%3D20250819040000%26et%3D20250819120000%26v%3D40.42532%2C-79.91643%2C9.233%2ClatLng%26ps%3D2400%26startDwell%3D0%26endDwell%3D0&&width=540&height=540

Last line

2025-08-20 07:45:03.735 THUMB 1715138:063161:0 request.body {"service":"Thumbnails","datetime":"2025-08-20T07:45:03-04:00","host":"-","level":"up","summary":"Last thumbnail failed","details":null,"payload":{},"valid_for_secs":null}
IP: 128.2.218.94 Browser: Python-urllib/3.7

Root

https://headless.earthtime.org/#l=mb_labeled,smell_my_city_pgh_reports_top2025,plume_202508190400_202508200400&bt=20250819040000&et=20250819120000&v=40.42532,-79.91643,9.233,latLng&ps=2400&startDwell=0&endDwell=0

Delegated to

https://hal40-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fheadless.earthtime.org%2F%23l%3Dmb_labeled%2Csmell_my_city_pgh_reports_top2025%2Cplume_202508190400_202508200400%26bt%3D20250819040000%26et%3D20250819120000%26v%3D40.42532%2C-79.91643%2C9.233%2ClatLng%26ps%3D2400%26startDwell%3D0%26endDwell%3D0&&width=540&height=540&format=zip&fps=30&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&disableUI&redo=0&workerepoch=1755690296214

Raw stats

{"REMOTE_ADDR": "128.2.218.94", "HTTP_USER_AGENT": "Python-urllib/3.7", "delegatedTo": "https://hal40-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fheadless.earthtime.org%2F%23l%3Dmb_labeled%2Csmell_my_city_pgh_reports_top2025%2Cplume_202508190400_202508200400%26bt%3D20250819040000%26et%3D20250819120000%26v%3D40.42532%2C-79.91643%2C9.233%2ClatLng%26ps%3D2400%26startDwell%3D0%26endDwell%3D0&&width=540&height=540&format=zip&fps=30&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&disableUI&redo=0&workerepoch=1755690296214", "FATALERROR": "Thumbnail delegation to hal40-thumbnailworker.createlab.org failed with status code 500 and response body <!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>500 Internal Server Error</title>\n</head><body>\n<h1>Internal Server Error</h1>\n<p>The server encountered an internal error or\nmisconfiguration and was unable to complete\nyour request.</p>\n<p>Please contact the server administrator at \n [no address given] to inform them of the time this error occurred,\n and the actions you performed just before this error.</p>\n<p>More information about this error may be available\nin the server error log.</p>\n<hr>\n<address>Apache/2.4.41 (Ubuntu) Server at hal40-thumbnailworker.createlab.org Port 443</address>\n</body></html>\n\n/t/thumbnails-v2.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1095:in `delegate_thumbnail'\n/t/thumbnails-v2.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1235:in `serve_thumbnail'\n/t/thumbnails-v2.createlab.org/timemachine-th"}
More: grep 1715138:063161 /t/thumbnails-v2.createlab.org/log.txt

Full log

2025-08-20 07:44:56.213 THUMB 1715138:063161:0 Not found in cache; computing
2025-08-20 07:44:56.213 THUMB 1715138:063161:0 STARTTHUMBNAIL https://thumbnails-earthtime.cmucreatelab.org/thumbnail?root=https%3A%2F%2Fheadless.earthtime.org%2F%23l%3Dmb_labeled%2Csmell_my_city_pgh_reports_top2025%2Cplume_202508190400_202508200400%26bt%3D20250819040000%26et%3D20250819120000%26v%3D40.42532%2C-79.91643%2C9.233%2ClatLng%26ps%3D2400%26startDwell%3D0%26endDwell%3D0&&width=540&height=540&format=zip&fps=30&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&disableUI&redo=0
2025-08-20 07:44:56.213 THUMB 1715138:063161:0 CHECKPOINTTHUMBNAIL START {"REMOTE_ADDR":"128.2.218.94","HTTP_USER_AGENT":"Python-urllib/3.7"}
2025-08-20 07:44:56.214 THUMB 1715138:063161:0 Captured resource lock /t/thumbnails-v2.createlab.org/locks/0049+hal40-thumbnailworker.createlab.org from /t/thumbnails-v2.createlab.org/locks
2025-08-20 07:44:56.214 THUMB 1715138:063161:0 Thumbnail worker hal40-thumbnailworker.createlab.org
2025-08-20 07:44:56.214 THUMB 1715138:063161:0 CHECKPOINTTHUMBNAIL DELEGATION_START {"REMOTE_ADDR":"128.2.218.94","HTTP_USER_AGENT":"Python-urllib/3.7","delegatedTo":"https://hal40-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fheadless.earthtime.org%2F%23l%3Dmb_labeled%2Csmell_my_city_pgh_reports_top2025%2Cplume_202508190400_202508200400%26bt%3D20250819040000%26et%3D20250819120000%26v%3D40.42532%2C-79.91643%2C9.233%2ClatLng%26ps%3D2400%26startDwell%3D0%26endDwell%3D0&&width=540&height=540&format=zip&fps=30&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&disableUI&redo=0&workerepoch=1755690296214"}
2025-08-20 07:44:56.214 THUMB 1715138:063161:0 Delegating thumbnail to hal40-thumbnailworker.createlab.org
2025-08-20 07:45:03.715 THUMB 1715138:063161:0 Thumbnail returned from hal40-thumbnailworker.createlab.org, status code 500, body length 633
2025-08-20 07:45:03.716 THUMB 1715138:063161:0 CHECKPOINTTHUMBNAIL DELEGATION_FINISHED {"REMOTE_ADDR":"128.2.218.94","HTTP_USER_AGENT":"Python-urllib/3.7","delegatedTo":"https://hal40-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fheadless.earthtime.org%2F%23l%3Dmb_labeled%2Csmell_my_city_pgh_reports_top2025%2Cplume_202508190400_202508200400%26bt%3D20250819040000%26et%3D20250819120000%26v%3D40.42532%2C-79.91643%2C9.233%2ClatLng%26ps%3D2400%26startDwell%3D0%26endDwell%3D0&&width=540&height=540&format=zip&fps=30&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&disableUI&redo=0&workerepoch=1755690296214"}
2025-08-20 07:45:03.716 THUMB 1715138:063161:0 CHECKPOINTTHUMBNAIL FATALERROR {"REMOTE_ADDR":"128.2.218.94","HTTP_USER_AGENT":"Python-urllib/3.7","delegatedTo":"https://hal40-thumbnailworker.createlab.org/thumbnail?root=https%3A%2F%2Fheadless.earthtime.org%2F%23l%3Dmb_labeled%2Csmell_my_city_pgh_reports_top2025%2Cplume_202508190400_202508200400%26bt%3D20250819040000%26et%3D20250819120000%26v%3D40.42532%2C-79.91643%2C9.233%2ClatLng%26ps%3D2400%26startDwell%3D0%26endDwell%3D0&&width=540&height=540&format=zip&fps=30&tileFormat=mp4&startDwell=0&endDwell=0&fromScreenshot&disableUI&redo=0&workerepoch=1755690296214","FATALERROR":"Thumbnail delegation to hal40-thumbnailworker.createlab.org failed with status code 500 and response body <!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>500 Internal Server Error</title>\n</head><body>\n<h1>Internal Server Error</h1>\n<p>The server encountered an internal error or\nmisconfiguration and was unable to complete\nyour request.</p>\n<p>Please contact the server administrator at \n [no address given] to inform them of the time this error occurred,\n and the actions you performed just before this error.</p>\n<p>More information about this error may be available\nin the server error log.</p>\n<hr>\n<address>Apache/2.4.41 (Ubuntu) Server at hal40-thumbnailworker.createlab.org Port 443</address>\n</body></html>\n\n/t/thumbnails-v2.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1095:in `delegate_thumbnail'\n/t/thumbnails-v2.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1235:in `serve_thumbnail'\n/t/thumbnails-v2.createlab.org/timemachine-thumbnail-server/thumbnail-server.rb:1360:in `<main>'"}
2025-08-20 07:45:03.716 THUMB 1715138:063161:0 url is https://stat.createlab.org/api/log host stat.createlab.org port 443
2025-08-20 07:45:03.716 THUMB 1715138:063161:0 request.body {"service":"Thumbnails","datetime":"2025-08-20T07:45:03-04:00","host":"-","level":"info","summary":"Thumbnail failed 1715138:063161 on thumbnails-earthtime.cmucreatelab.org in 7.5 seconds","details":"<a href=\"https://hal40-thumbnailworker.createlab.org/status?id=1715138:063161\">More info</a>","payload":{},"valid_for_secs":null}
2025-08-20 07:45:03.735 THUMB 1715138:063161:0 url is https://stat.createlab.org/api/log host stat.createlab.org port 443
2025-08-20 07:45:03.735 THUMB 1715138:063161:0 request.body {"service":"Thumbnails","datetime":"2025-08-20T07:45:03-04:00","host":"-","level":"up","summary":"Last thumbnail failed","details":null,"payload":{},"valid_for_secs":null}