sbrl-github/research-rainfallradar
1
0
Fork 0
mirror of https://github.com/sbrl/research-rainfallradar synced 2024-12-23 06:35:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
629 commits 1 branch 0 tags 2.5 MiB
Commit graph

629 commits

This branch
This branch
All branches
Author SHA1 Message Date
Starbeamrainbowlabs
0761651ccf
tvt: implement CallbackExtraValidation, which allows for a third split 
it should tie into Tensorflow's logging  just fine so long as it's the first callback in the queue.

***** TEST SCRIPT *****

model = tf.keras.Sequential([
    tf.keras.layers.Dense(64, activation='relu', input_shape=(10,)),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.Dense(1)
])

model.compile(optimizer='adam', loss='mse', metrics=['mae'])
X = np.random.random((100, 10))
y = np.random.random((100, 1))

split = 80
X_train, X_val = X[:split], X[split:]
y_train, y_val = y[:split], y[split:]

train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train)).batch(10)
val_dataset = tf.data.Dataset.from_tensor_slices((X_val, y_val)).batch(10)

history = model.fit(train_dataset,
	epochs=10,
	validation_data=val_dataset,
	callbacks=[
		CallbackExtraValidation({
			"test": val_dataset
		}, verbose=0),
		tf.keras.callbacks.CSVLogger("/dev/stdout", separator="\t")
	],
	verbose=0
)

print(f"DEBUG history {history}")
 2024-08-30 18:07:17 +01:00
Starbeamrainbowlabs
b5310304bd
wrangler: update dependencies 2024-08-29 19:38:02 +01:00
Starbeamrainbowlabs
e5f6e6394f
Implement initial UNTESTED support for split_validation and split_test 2024-08-29 19:33:40 +01:00
Starbeamrainbowlabs
b2b96ab636
replace tensorflow_addons with keras_cv 2024-08-29 19:32:54 +01:00
Starbeamrainbowlabs
2d73d84e20
uv pip sync doesn't install dependencies of packages 2024-08-29 19:31:42 +01:00
Starbeamrainbowlabs
b31485f7a1
add pyproject.toml 2024-08-29 18:58:57 +01:00
Starbeamrainbowlabs
61ee2326cf
licence change 2024-08-29 18:36:11 +01:00
Starbeamrainbowlabs
18fd4b5d4f
Correct github language bar; Jupyter notebook are not part of the main project 2024-08-29 18:32:28 +01:00
Starbeamrainbowlabs
c98c42fa7e
dataset_mono: Implement validate_percentage + test_percentage support 
This removes the train_percentage argument

TODO: map this forwards to enable support in deeplabv3_plus_test_rainfall

...thinking about it, it's really not a test now, is it? Updating the filename would be such a /hassle/ though....
 2024-08-29 16:44:19 +01:00
Starbeamrainbowlabs
5d62e3cee8
Implement env from PhD-smflooding-scene 2024-08-29 16:43:29 +01:00
Starbeamrainbowlabs
a75d4f5d79
dataset_mono: add todo 2024-08-02 19:48:29 +01:00
Starbeamrainbowlabs
e15f6cd954
Update README w/proper paper link 2024-06-21 16:03:58 +01:00
Starbeamrainbowlabs
d22dd6ff18
README: more tidy up 2023-11-30 17:08:50 +00:00
Starbeamrainbowlabs
1cbb241786
README: tidy up 2023-11-30 17:02:08 +00:00
Starbeamrainbowlabs
844d8e6dd4
rw: update dependencies 2023-11-30 17:01:13 +00:00
Starbeamrainbowlabs
c5be527ffb
rw/README: fill out 2023-11-30 16:59:52 +00:00
Starbeamrainbowlabs
4626549352
rw/recompress: clarify subcommand description 2023-11-30 16:59:45 +00:00
Starbeamrainbowlabs
18195c2d15
rw: mark as 'private' in package.json to avoid accidental publication 2023-11-30 16:59:31 +00:00
Starbeamrainbowlabs
002203d20b
vscodium/settings: spellings 2023-11-30 16:59:06 +00:00
Starbeamrainbowlabs
294fde7c82
rw; add requirements.txt 2023-11-30 16:45:04 +00:00
Starbeamrainbowlabs
92fc34ebb1
rw/child_process:json2tfrecord: Add RAINFALL_MAX_NUMBER env var 
use use you'd need to export RAINFALL_MAX_NUMBER=some_integer_value, since it is used by a subprocess rather than the main Node.js process itself
 2023-11-30 16:44:43 +00:00
Starbeamrainbowlabs
0f9f185983
dlr: add PARALLEL_READS env var, update docs 2023-11-30 16:33:22 +00:00
Starbeamrainbowlabs
60674fb6b3
README: finish filling out 2023-11-30 16:32:46 +00:00
Starbeamrainbowlabs
13ae4df725
vscodium/settings: spellings 2023-11-30 16:32:17 +00:00
Starbeamrainbowlabs
df621fd7d2
README: Continue filling out, but we're not there yet. 2023-11-29 17:31:47 +00:00
Starbeamrainbowlabs
3eec01a28c
README: update image 2023-11-29 15:16:01 +00:00
Starbeamrainbowlabs
a1a2b105e6
README: Add under construction notice 2023-11-29 15:15:07 +00:00
Starbeamrainbowlabs
1d8807793f
README: start expanding, buti ts not finished yet 2023-11-28 16:46:02 +00:00
Starbeamrainbowlabs
e14fa275ab
rr de: rerun to recaalculate 
these values are accurate to our dataset now
 2023-11-23 18:35:39 +00:00
Starbeamrainbowlabs
76b2956703
rr de: update to dataset_mono_predict 2023-11-03 15:48:53 +00:00
Starbeamrainbowlabs
cbb916327b
vscode → settings.json: spellings 2023-10-26 17:12:46 +01:00
Starbeamrainbowlabs
e44b5533b1
plot_metrics_multi: add option to plot train/val separately 2023-07-20 15:49:09 +01:00
Starbeamrainbowlabs
b5b26d980b
dlr: print filepaths 
we need to know which is which with  this seed so we can visualise for the paper
 2023-07-13 19:58:38 +01:00
Starbeamrainbowlabs
d7fdf8f038
README: fix diagram 2023-07-06 15:31:13 +01:00
Starbeamrainbowlabs
d93365a191
Update license to AGPL. 2023-07-06 15:16:11 +01:00
Starbeamrainbowlabs
636b316bfc
plot_multi: actually fix the plots 2023-07-05 16:35:26 +01:00
Starbeamrainbowlabs
9efc72db73
dlr ds/mono: just why 2023-06-16 18:36:18 +01:00
Starbeamrainbowlabs
a4e80229fb
dlr ds/mono: fixup 2023-06-16 18:35:02 +01:00
Starbeamrainbowlabs
7869505cfb
dlr: add PREDICT_AS_ONE 2023-06-16 18:23:40 +01:00
Starbeamrainbowlabs
4bbc4c29c4
plot_metrics_multi: add RESOLUTION env var 2023-06-14 15:50:37 +01:00
Starbeamrainbowlabs
18db54f0a7
plot_metrics_multi: FINALLY write a script to generate charts for multiple models at the same time 2023-06-08 18:13:20 +01:00
Starbeamrainbowlabs
fa37185950
plot_metrics: improve CLI 2023-06-08 15:20:32 +01:00
Starbeamrainbowlabs
3be38823db
dlr: fix another crash 2023-05-19 22:00:23 +01:00
Starbeamrainbowlabs
3e42972fb0
dlr: add comment 2023-05-11 17:45:52 +01:00
Starbeamrainbowlabs
fe05dd33fc
dlr: missing quotes 2023-05-11 16:23:36 +01:00
Starbeamrainbowlabs
5a73388a80
dlr: add RANDSEED to slurm 2023-05-11 16:02:13 +01:00
Starbeamrainbowlabs
ae0c672f60
Merge branch 'main' of git.starbeamrainbowlabs.com:sbrl/PhD-Rainfall-Radar 2023-05-11 15:59:08 +01:00
Starbeamrainbowlabs
20092c6829
shuffle: add random seed env var 2023-05-11 15:59:01 +01:00
Starbeamrainbowlabs
535bd6f579
dlr: fix crash 2023-05-07 19:00:02 +01:00
Starbeamrainbowlabs
ac040717e6
autoset the value of factor 2023-05-04 19:57:02 +01:00