Function to write cached arrow files. Writes to the first defined storage in config::get("tessilake"),
and then syncs to the other locations by calling sync_cache.
Usage
write_cache(
x,
table_name,
type,
depth = deprecated(),
incremental = FALSE,
sync = TRUE,
...
)
cache_write(
x,
table_name,
depth,
type,
primary_keys = cache_get_attributes(x)$primary_keys,
partition = !is.null(primary_keys),
overwrite = FALSE,
num_tries = 60,
...
)Arguments
- x
data.frame to be written
- table_name
string
- type
string, e.g. "tessi" or "stream"
- depth
string, e.g. "deep" or "shallow", deprecated in write_cache
- incremental
boolean, whether to call cache_update or cache_write to update the cached dataset.
- sync
boolean, whether or not to sync the written cache to other storages
- ...
extra arguments passed on to arrow::write_feather, arrow::write_parquet or arrow::write_dataset
- primary_keys
character vector of columns to be used as primary keys
- partition
boolean or character, if TRUE, partition is derived from primary_keys; if character, partition identifies the column to use for partitioning
- overwrite
boolean, whether or not to overwrite an existing cache
- num_tries
integer number of times to try reading before failing
Functions
cache_write(): Underlying cache writer that invokes arrow::write_feather, arrow::write_parquet or arrow::write_dataset and handles partitioning
Examples
if (FALSE) { # \dontrun{
x <- data.table(a = c(1, 2, 3))
write_cache(x, "test", "stream", primary_keys = c("a"))
} # }