158 commits

Author	SHA1	Message	Date
Starbeamrainbowlabs	1103ae5561	ai: tweak progress bar	2022-09-16 16:07:16 +01:00
Starbeamrainbowlabs	1e35802d2b	ai: fix embed i/o	2022-09-16 16:02:27 +01:00
Starbeamrainbowlabs	ed94da7492	fixup	2022-09-16 15:51:26 +01:00
Starbeamrainbowlabs	366db658a8	ds predict: fix filenames in	2022-09-16 15:45:22 +01:00
Starbeamrainbowlabs	e333dcba9c	tweak projection head	2022-09-16 15:36:01 +01:00
Starbeamrainbowlabs	6defd24000	bugfix: too many values to unpack	2022-09-15 19:56:17 +01:00
Starbeamrainbowlabs	e3c8277255	ai: tweak the segmentation model structure	2022-09-15 19:54:50 +01:00
Starbeamrainbowlabs	1bc8a5bf13	ai: fix crash	2022-09-15 19:37:06 +01:00
Starbeamrainbowlabs	bd64986332	ai: implement batched_iterator to replace .batch() ... ...apparently .batch() means you get a BatchedDataset or whatever when you iterate it like a tf.function instead of the actual tensor :-/	2022-09-15 19:16:38 +01:00
Starbeamrainbowlabs	ccd256c00a	embed rainfall radar, not both	2022-09-15 17:37:04 +01:00
Starbeamrainbowlabs	2c74676902	predict → predict_on_batch	2022-09-15 17:31:50 +01:00
Starbeamrainbowlabs	f036e79098	fixup	2022-09-15 17:09:26 +01:00
Starbeamrainbowlabs	d5f1a26ba3	disable prefetching when predicting a thing	2022-09-15 17:09:09 +01:00
Starbeamrainbowlabs	8770638022	ai: call .predict(), not the model itself	2022-09-14 17:41:01 +01:00
Starbeamrainbowlabs	a96cefde62	ai: predict oops	2022-09-14 17:37:48 +01:00
Starbeamrainbowlabs	fa3165a5b2	dataset: simplify dataset_predict	2022-09-14 17:33:17 +01:00
Starbeamrainbowlabs	279e27c898	fixup	2022-09-14 17:16:49 +01:00
Starbeamrainbowlabs	fad3313ede	fixup	2022-09-14 17:14:04 +01:00
Starbeamrainbowlabs	1e682661db	ai: kwargs in from_checkpoint	2022-09-14 17:11:06 +01:00
Starbeamrainbowlabs	6bda24d4da	ai: how did I miss that?! ... bugfix ah/c	2022-09-14 16:53:43 +01:00
Starbeamrainbowlabs	decdd434d8	ai from_checkpoint: bugfix	2022-09-14 16:49:01 +01:00
Starbeamrainbowlabs	c9e00ea485	pretrain_predict: fix import	2022-09-14 16:02:36 +01:00
Starbeamrainbowlabs	f568d8d19f	io.open → handle_open ... this was we get transparent .gz support	2022-09-14 16:01:22 +01:00
Starbeamrainbowlabs	9b25186541	fix --only-gpu	2022-09-14 15:55:21 +01:00
Starbeamrainbowlabs	a9c9c70d13	typo	2022-09-14 15:17:59 +01:00
Starbeamrainbowlabs	fb8f884487	add umap dependencies	2022-09-14 15:16:45 +01:00
Starbeamrainbowlabs	1876a8883c	ai pretrain-predict: fix - → _ in cli parsing	2022-09-14 15:12:07 +01:00
Starbeamrainbowlabs	f97b771922	make --help display help	2022-09-14 15:03:07 +01:00
Starbeamrainbowlabs	3c1aef5913	update help	2022-09-13 19:36:56 +01:00
Starbeamrainbowlabs	206257f9f5	ai pretrain_predict: no need to plot here anymore	2022-09-13 19:35:44 +01:00
Starbeamrainbowlabs	7685ec3e8b	implement ability to embed & plot pretrained embeddings	2022-09-13 19:18:59 +01:00
Starbeamrainbowlabs	7130c4fdf8	start implementing core image segmentation model	2022-09-07 17:45:38 +01:00
Starbeamrainbowlabs	22620a1854	ai: implement saving only the rainfall encoder	2022-09-06 19:48:46 +01:00
Starbeamrainbowlabs	4c4358c3e5	whitespace	2022-09-06 16:24:11 +01:00
Starbeamrainbowlabs	4202821d98	typo	2022-09-06 15:37:36 +01:00
Starbeamrainbowlabs	3e13ad12c8	ai Bugfix LayerContrastiveEncoder: channels → input_channels ... for consistency	2022-09-05 23:53:16 +01:00
Starbeamrainbowlabs	ead8009425	pretrain: add CLI arg for size of watch prediction width/height	2022-09-05 15:36:40 +01:00
Starbeamrainbowlabs	9d39215dd5	dataset: drop incomplete batches	2022-09-05 15:36:10 +01:00
Starbeamrainbowlabs	c94f5d042e	ai: slurm fixup again	2022-09-02 19:13:56 +01:00
Starbeamrainbowlabs	7917820e59	ai: slurm fixup	2022-09-02 19:11:54 +01:00
Starbeamrainbowlabs	cd104190e8	slurm: no need to be exclusive anymore	2022-09-02 19:09:45 +01:00
Starbeamrainbowlabs	a9dede7bfe	ai: n=10 slurm	2022-09-02 19:09:07 +01:00
Starbeamrainbowlabs	42de502f99	slurm-pretrain: set name	2022-09-02 19:08:16 +01:00
Starbeamrainbowlabs	457c2eef0d	ai: fixup	2022-09-02 19:06:54 +01:00
Starbeamrainbowlabs	a3f03b6d8d	slurm-pretrain: prepare	2022-09-02 19:05:18 +01:00
Starbeamrainbowlabs	1d1533d160	ai: how did things get this confusing	2022-09-02 18:51:46 +01:00
Starbeamrainbowlabs	1c5defdcd6	ai: cats	2022-09-02 18:45:23 +01:00
Starbeamrainbowlabs	6135bcd0cd	fixup	2022-09-02 18:41:31 +01:00
Starbeamrainbowlabs	c33c0a0899	ai: these shapes are so annoying	2022-09-02 18:39:24 +01:00
Starbeamrainbowlabs	88acd54a97	ai: off-by-one	2022-09-02 18:13:48 +01:00
Starbeamrainbowlabs	23bcd294c4	AI: 0.75 → 0.5?	2022-09-02 18:12:51 +01:00
Starbeamrainbowlabs	b0e1aeac35	ai: knew it	2022-09-02 18:10:29 +01:00
Starbeamrainbowlabs	ef0f15960d	send correct water shape to	2022-09-02 18:09:36 +01:00
Starbeamrainbowlabs	ad156a9a00	ai dataset: centre crop the water data to 75% original size ... this should both help the model and reduce memory usage	2022-09-02 18:05:32 +01:00
Starbeamrainbowlabs	efe41b96ec	fixup	2022-09-02 17:58:02 +01:00
Starbeamrainbowlabs	f8ee0afca1	ai: fix arch_name plumbing	2022-09-02 17:57:07 +01:00
Starbeamrainbowlabs	3d44831080	Add NO_PREFETCH env var	2022-09-02 17:55:04 +01:00
Starbeamrainbowlabs	3e0ca6a315	ai: fix summary file writing; make water encoder smaller	2022-09-02 17:51:45 +01:00
Starbeamrainbowlabs	389216b391	reordering	2022-09-02 17:31:19 +01:00
Starbeamrainbowlabs	b9d01ddadc	summary logger → summarywriter	2022-09-02 17:28:00 +01:00
Starbeamrainbowlabs	9f7f4af784	dataset: properly resize	2022-09-02 17:09:38 +01:00
Starbeamrainbowlabs	c78b6562ef	debug adjustment	2022-09-02 17:00:21 +01:00
Starbeamrainbowlabs	c89677abd7	dataset: explicit reshape	2022-09-02 16:57:59 +01:00
Starbeamrainbowlabs	c066ea331b	dataset: namespace → dict ... Python is so annoying	2022-09-02 16:07:44 +01:00
Starbeamrainbowlabs	dbff15d4a9	add bug comment	2022-09-01 19:05:50 +01:00
Starbeamrainbowlabs	3e4128c0a8	resize rainfall to be 1/2 size of current	2022-09-01 18:47:07 +01:00
Starbeamrainbowlabs	8a86728b54	ai: how did I miss that	2022-09-01 18:07:53 +01:00
Starbeamrainbowlabs	b2f1ba29bb	debug a	2022-09-01 18:05:33 +01:00
Starbeamrainbowlabs	e2f621e212	even moar debug logging	2022-09-01 17:49:51 +01:00
Starbeamrainbowlabs	2663123407	moar logging	2022-09-01 17:40:23 +01:00
Starbeamrainbowlabs	c2fcb3b954	ai: channels_first → channels_last	2022-09-01 17:06:18 +01:00
Starbeamrainbowlabs	f1d7973f22	ai: add dummy label	2022-09-01 17:01:00 +01:00
Starbeamrainbowlabs	17d42fe899	ai: json.dumps	2022-09-01 16:21:52 +01:00
Starbeamrainbowlabs	940f7aa1b5	ai: set self.model	2022-09-01 16:20:23 +01:00
Starbeamrainbowlabs	f1be5fe2bd	ai: add missing arguments to LossContrastive	2022-09-01 16:14:00 +01:00
Starbeamrainbowlabs	ddb375e906	ai: another typo	2022-09-01 16:12:05 +01:00
Starbeamrainbowlabs	cb3e1d3a23	ai: fixup	2022-09-01 16:10:50 +01:00
Starbeamrainbowlabs	e4c95bc7e3	typo	2022-09-01 16:09:24 +01:00
Starbeamrainbowlabs	cfbbe8e8cf	ai: global? really?	2022-09-01 16:06:24 +01:00
Starbeamrainbowlabs	4952ead094	ai: try the nonlocal keyword? I'm unsure what's going on here ... ....clearly I need to read up on scoping in python.	2022-09-01 16:02:37 +01:00
Starbeamrainbowlabs	1d9d0fbb73	requirements.txt:a dd tensorflow_addons	2022-09-01 15:56:02 +01:00
Starbeamrainbowlabs	8bdded23eb	ai: fix 'nother crash' name ConvNeXt submodels	2022-08-31 18:57:27 +01:00
Starbeamrainbowlabs	b2a320134e	ai: typo	2022-08-31 18:54:03 +01:00
Starbeamrainbowlabs	e4edc68df5	ai: add missing gamma layer	2022-08-31 18:52:35 +01:00
Starbeamrainbowlabs	51cf08a386	ResNetRSV2 → ConvNeXt ... ironically this makes the model simpler o/	2022-08-31 18:51:01 +01:00
Starbeamrainbowlabs	3d614d105b	channels first	2022-08-31 18:06:59 +01:00
Starbeamrainbowlabs	654eefd9ca	properly handle water dimensions; add log files to .gitignore ... TODO: add heightmap	2022-08-31 18:03:39 +01:00
Starbeamrainbowlabs	5846828f9e	debug logging	2022-08-31 17:48:09 +01:00
Starbeamrainbowlabs	12c77e128d	handle feature_dim properly	2022-08-31 17:41:51 +01:00
Starbeamrainbowlabs	c52a9f961c	and another	2022-08-31 17:37:28 +01:00
Starbeamrainbowlabs	58dbabd561	fix another crash	2022-08-31 17:33:07 +01:00
Starbeamrainbowlabs	5fc0725917	slurm: chmod +x	2022-08-31 16:33:07 +01:00
Starbeamrainbowlabs	dbf929325a	typo; add pretrain slurm job file	2022-08-31 16:32:17 +01:00
Starbeamrainbowlabs	e0162bc70b	requirements.txt: add missing dependencies	2022-08-31 16:25:47 +01:00
Starbeamrainbowlabs	f2312c1184	fix crash	2022-08-31 16:25:27 +01:00
Starbeamrainbowlabs	15a3519107	ai: the best thing about implementing a model is that you don't have to test it on the same day :P	2022-08-11 18:26:28 +01:00
Starbeamrainbowlabs	c0a9cb12d8	ai: start creating initial model implementation. ... it's not hooked up to the CLI yet though. Focus is still on ensuring the dataset is in the right format though	2022-08-10 19:03:25 +01:00
Starbeamrainbowlabs	b52c7f89a7	Move dataset parsing function to the right place	2022-08-10 17:24:55 +01:00
Starbeamrainbowlabs	222a6146ec	write glue for .jsonl.gz → .tfrecord.gz converter	2022-08-08 15:33:59 +01:00
Starbeamrainbowlabs	28a3f578d5	update .gitignore	2022-08-04 16:49:53 +01:00