research-rainfallradar/aimodel
Starbeamrainbowlabs 0761651ccf
tvt: implement CallbackExtraValidation, which allows for a third split
it should tie into Tensorflow's logging  just fine so long as it's the first callback in the queue.

***** TEST SCRIPT *****

model = tf.keras.Sequential([
    tf.keras.layers.Dense(64, activation='relu', input_shape=(10,)),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.Dense(1)
])

model.compile(optimizer='adam', loss='mse', metrics=['mae'])
X = np.random.random((100, 10))
y = np.random.random((100, 1))

split = 80
X_train, X_val = X[:split], X[split:]
y_train, y_val = y[:split], y[split:]

train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train)).batch(10)
val_dataset = tf.data.Dataset.from_tensor_slices((X_val, y_val)).batch(10)

history = model.fit(train_dataset,
	epochs=10,
	validation_data=val_dataset,
	callbacks=[
		CallbackExtraValidation({
			"test": val_dataset
		}, verbose=0),
		tf.keras.callbacks.CSVLogger("/dev/stdout", separator="\t")
	],
	verbose=0
)

print(f"DEBUG history {history}")
2024-08-30 18:07:17 +01:00
..
src tvt: implement CallbackExtraValidation, which allows for a third split 2024-08-30 18:07:17 +01:00
pyproject.toml uv pip sync doesn't install dependencies of packages 2024-08-29 19:31:42 +01:00
requirements.txt uv pip sync doesn't install dependencies of packages 2024-08-29 19:31:42 +01:00
slurm-encoderonly-rainfall.job weird, XLA_FLAGS cuda data dir wasn't needed before 2023-02-10 13:28:34 +00:00
slurm-pretrain-plot.job slurm: customise log file names 2022-11-10 21:09:34 +00:00
slurm-pretrain-predict.job slurm: customise log file names 2022-11-10 21:09:34 +00:00
slurm-pretrain.job slurm: customise log file names 2022-11-10 21:09:34 +00:00
slurm-TEST-deeplabv3p-rainfall.job Implement initial UNTESTED support for split_validation and split_test 2024-08-29 19:33:40 +01:00
slurm-TEST-deeplabv3p.job slurm: add job file for deeplabv3 test 2022-12-12 19:31:49 +00:00
slurm-train-mono.job slurm-train-mono: add HEIGHTMAP 2022-12-06 19:28:06 +00:00
slurm-train.job slurm-train: reduce RAM required by 10GB 2022-11-29 19:15:34 +00:00
uv.lock add pyproject.toml 2024-08-29 18:58:57 +01:00