80 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Starbeamrainbowlabs	e4c95bc7e3	typo	2022-09-01 16:09:24 +01:00
Starbeamrainbowlabs	cfbbe8e8cf	ai: global? really?	2022-09-01 16:06:24 +01:00
Starbeamrainbowlabs	4952ead094	ai: try the nonlocal keyword? I'm unsure what's going on here ... ....clearly I need to read up on scoping in python.	2022-09-01 16:02:37 +01:00
Starbeamrainbowlabs	1d9d0fbb73	requirements.txt:a dd tensorflow_addons	2022-09-01 15:56:02 +01:00
Starbeamrainbowlabs	8bdded23eb	ai: fix 'nother crash' name ConvNeXt submodels	2022-08-31 18:57:27 +01:00
Starbeamrainbowlabs	b2a320134e	ai: typo	2022-08-31 18:54:03 +01:00
Starbeamrainbowlabs	e4edc68df5	ai: add missing gamma layer	2022-08-31 18:52:35 +01:00
Starbeamrainbowlabs	51cf08a386	ResNetRSV2 → ConvNeXt ... ironically this makes the model simpler o/	2022-08-31 18:51:01 +01:00
Starbeamrainbowlabs	3d614d105b	channels first	2022-08-31 18:06:59 +01:00
Starbeamrainbowlabs	654eefd9ca	properly handle water dimensions; add log files to .gitignore ... TODO: add heightmap	2022-08-31 18:03:39 +01:00
Starbeamrainbowlabs	5846828f9e	debug logging	2022-08-31 17:48:09 +01:00
Starbeamrainbowlabs	12c77e128d	handle feature_dim properly	2022-08-31 17:41:51 +01:00
Starbeamrainbowlabs	c52a9f961c	and another	2022-08-31 17:37:28 +01:00
Starbeamrainbowlabs	58dbabd561	fix another crash	2022-08-31 17:33:07 +01:00
Starbeamrainbowlabs	5fc0725917	slurm: chmod +x	2022-08-31 16:33:07 +01:00
Starbeamrainbowlabs	dbf929325a	typo; add pretrain slurm job file	2022-08-31 16:32:17 +01:00
Starbeamrainbowlabs	e0162bc70b	requirements.txt: add missing dependencies	2022-08-31 16:25:47 +01:00
Starbeamrainbowlabs	f2312c1184	fix crash	2022-08-31 16:25:27 +01:00
Starbeamrainbowlabs	fe7a8b3fc0	Add 9 simple steps to use the rainfall radar model..... oh boy.	2022-08-31 16:18:28 +01:00
Starbeamrainbowlabs	15a3519107	ai: the best thing about implementing a model is that you don't have to test it on the same day :P	2022-08-11 18:26:28 +01:00
Starbeamrainbowlabs	28bcdf2192	vscode settings file	2022-08-11 16:38:25 +01:00
Starbeamrainbowlabs	c0a9cb12d8	ai: start creating initial model implementation. ... it's not hooked up to the CLI yet though. Focus is still on ensuring the dataset is in the right format though	2022-08-10 19:03:25 +01:00
Starbeamrainbowlabs	6cdf2b2389	wrangler python child: explicitly close stdout+stderr. ... Hopefully this will avoid any more hanging issues.	2022-08-10 18:51:30 +01:00
Starbeamrainbowlabs	5880bf9020	wrangler: add current date to process indicator. ... There's a bug that causes it to hang, but we don't know why	2022-08-10 18:50:57 +01:00
Starbeamrainbowlabs	231c832888	wrangler bugfix: crashes; logging output	2022-08-10 17:33:10 +01:00
Starbeamrainbowlabs	b52c7f89a7	Move dataset parsing function to the right place	2022-08-10 17:24:55 +01:00
Starbeamrainbowlabs	50f214450f	wrangler: fix crash	2022-08-10 17:05:01 +01:00
Starbeamrainbowlabs	0bac8c8c0c	fixup	2022-08-08 17:23:24 +01:00
Starbeamrainbowlabs	405f1a0bb0	fixup	2022-08-08 17:22:31 +01:00
Starbeamrainbowlabs	5e1356513c	slurm: use compute, because 28 tf processes in parallel is too much for the GPU memory	2022-08-08 17:22:18 +01:00
Starbeamrainbowlabs	133ef59af3	fixup	2022-08-08 16:33:05 +01:00
Starbeamrainbowlabs	80e1a33ee2	slurm-jsonl2tfrecord.job: auto install dependencies	2022-08-08 16:31:49 +01:00
Starbeamrainbowlabs	1442d20524	slurm: request gpu	2022-08-08 15:56:46 +01:00
Starbeamrainbowlabs	f6f2e3694c	json2tfrecord: write slurm job file	2022-08-08 15:53:32 +01:00
Starbeamrainbowlabs	222a6146ec	write glue for .jsonl.gz → .tfrecord.gz converter	2022-08-08 15:33:59 +01:00
Starbeamrainbowlabs	f3652edf82	fixup	2022-08-05 19:10:40 +01:00
Starbeamrainbowlabs	9399d1d8f5	Create (untested) JS interface to Python jsonl→tfrecord converter ... also test Python .jsonl.gz → .tfrecord.gz	2022-08-05 19:10:28 +01:00
Starbeamrainbowlabs	a02c3436ab	get python bridge working t convert .jsonl.gz → .tfrecord.gz	2022-08-05 18:07:04 +01:00
Starbeamrainbowlabs	28a3f578d5	update .gitignore	2022-08-04 16:49:53 +01:00
Starbeamrainbowlabs	2ccc1be414	json2tfrecord: write (untested python to convert .jsonl → .tfrecord	2022-07-28 19:48:25 +01:00
Starbeamrainbowlabs	323d708692	dataset: add todo ... just why, Tensorflow?! tf.data.TextLineDataset looks almost too good to be true..... and it is, as despite supporting decompressing via gzip(!) it doesn't look like we can convince it to parse JSON :-/	2022-07-26 19:53:18 +01:00
Starbeamrainbowlabs	b53c77a2cb	index.py: call static function name run	2022-07-26 19:51:28 +01:00
Starbeamrainbowlabs	a7ed58fc03	ai: move requirements.txt to the right place	2022-07-26 19:25:11 +01:00
Starbeamrainbowlabs	e93a95f1b3	ai dataset: add if main == main	2022-07-26 19:24:40 +01:00
Starbeamrainbowlabs	de4c3dab17	typo	2022-07-26 19:14:55 +01:00
Starbeamrainbowlabs	18a7d3674b	ai: create (untested) dataset	2022-07-26 19:14:10 +01:00
Starbeamrainbowlabs	dac6919fcd	ai: start creating initial scaffolding	2022-07-25 19:01:10 +01:00
Starbeamrainbowlabs	1ec502daea	Remove rogue package*.json files	2022-07-25 19:00:21 +01:00
Starbeamrainbowlabs	927c30e189	recompress files in the right order	2022-07-25 18:44:23 +01:00
Starbeamrainbowlabs	3332fa598a	Add new recompress subcommand ... also fix typos, CLI definitions	2022-07-25 17:54:23 +01:00