82d5c7f336
feat: support parallel, chunked upload via `influxdb_iox write` of line protocol, gzip'd line protocol, and parquet ( #5757 )
...
* feat: Upload in small chunks and in parallel
* fix: doclink
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: Update influxdb_iox_client/src/client/write.rs
* fix: fixup error handling and fmt
* fix: Make default chunk sizes the same and add docs
* fix: clippy
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2022-09-30 21:02:38 +00:00
cd4087e00d
style: add no todo!() or dbg!() lints
...
Some crates had theme, some not - lets be consistent and have the
compiler spot dbg!() and todo!() macro calls - they should never be in
prod code!
2022-09-29 13:10:07 +02:00
65f1550126
feat: Implement `debug parquet_to_lp` command to convert parquet to line protocol ( #5734 )
...
* feat: add `influxdb_iox debug parquet_to_lp` command
* chore: Run cargo hakari tasks
* fix: update command description
* fix: remove unecessary Result import
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2022-09-26 14:17:27 +00:00
3a501a4a10
fix: Remove an immediate ref to a deref
...
Caught by clippy now. https://rust-lang.github.io/rust-clippy/master/index.html#borrow_deref_ref
2022-08-11 15:04:14 -04:00
a4e2f880be
feat: Expose a C API for the IOx LP parser ( #5267 )
...
Can be useful to call the IOx LP parser from other processes, for example from Go.
I used it to run an online comparison of IOx and influxdb Go LP parser in order to identify compatibility
issues.
2022-08-02 15:44:41 +00:00
9c9658ca38
test(influxdb_line_protocol): add value verification test ( #5270 )
2022-08-02 11:18:09 +00:00
84a856069b
fix: Scientific notation without + or -
...
Closes #5264
2022-08-02 05:46:28 +02:00
a926996485
fix: Negative scientific notation without decimal parts
...
Closes #5263
2022-08-02 05:40:55 +02:00
de22b6b080
test: Add a test for LP quoting ( #5210 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-26 20:08:37 +00:00
c5451afa08
feat: Implement LP builder ( #5183 )
...
Helps with https://github.com/influxdata/influxrpc_compare/issues/179
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-22 14:41:10 +00:00
03aa338897
fix: Remove unnecessary String allocations found by clippy
2022-07-01 13:31:32 -04:00
71da70e6fe
test: cover all string field types
...
Adds tests that cover EscapedStr::SingleSlice in addition to the
existing EscapedStr::CopiedValue variants. Remove 1 space from assert.
2022-04-25 11:28:52 +01:00
680742103a
refactor: use std::mem::discriminant
2022-04-22 17:37:25 +01:00
cf200e509e
fix: handle LP lines with duplicated fields/tags
...
This commit allows the LP consumer to correctly handle line protocol
writes that duplicate one or more fields or tags within a single line:
table v=2,bananas=42,v=3,platanos=24
▲ ▲
└───────┬──────┘
│
duplicate field "v"
This change implements the following logic when processing fields:
IF field is duplicated
IF all duplicate field values are of the same data type
use last occurrence ("last write wins")
ELSE
return an error
Any duplication of tags is rejected, and use of a field name as both a
field and a tag is remains forbidden (unchanged in this PR, a previously
agreed breaking change from TSM).
See https://github.com/influxdata/influxdb_iox/issues/4326 for context.
2022-04-22 14:40:51 +01:00
33851be3a5
chore: upgrade Rust to 1.59 ( #3875 )
...
Mostly a few new clippy crates around `flat_map`, `and_then`, and
"underscore locks" (!!!):
https://rust-lang.github.io/rust-clippy/master/index.html#let_underscore_lock
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-28 15:14:19 +00:00
f3f6f335a9
chore: upgrade to snafu 0.7 ( #3440 )
2022-01-11 19:22:36 +00:00
368f0369ee
chore: Rust 1.55
2021-09-10 12:36:49 +02:00
1effb11ad9
refactor: move delete parseing work out of influxdb_line_protocol crate
2021-09-09 15:27:17 -04:00
b4f8fad400
refactor: address review comments and remove uno longer needed dependencies
2021-09-09 14:24:58 -04:00
00df7b064c
feat: finally have the delete predicate parsed
2021-09-08 17:30:10 -04:00
9ee1bdeeb9
refactor: address review comments
2021-09-07 10:24:38 -04:00
9de3b79a90
refactor: more cleanup
2021-09-06 01:45:47 -04:00
de0bd80c3d
refactor: cleanup
2021-09-06 01:07:07 -04:00
4801b2c238
feat: Have the ParseDelete message and its corresponding ProvidedParseDelete struct ready for building delete parser
2021-09-06 00:13:59 -04:00
d8b3208896
refactor: sstart building parser in the client side
2021-09-03 17:56:45 -04:00
bbc0fcf4c0
refactor: cleanup
2021-09-02 17:21:55 -04:00
e2274a9f41
feat: parser for delete predicate
2021-09-02 17:02:05 -04:00
efcdc8e50f
feat: implement delete parser
2021-09-01 17:31:49 -04:00
a4183de411
feat: more progress on the delete flow from grpc API to catalog chunks
2021-08-31 17:42:07 -04:00
3de6b44e23
build: use new rustdoc lint name ( #2261 )
...
* fix: nocache feature code rot
The MBChunk::snapshot code when using the "nocache" option no longer
compiles - this commit updates it to match the not(nocache) code.
* build: use updated broken_intra_doc_links name
The broken_intra_doc_links lint was renamed
rustdoc::broken_intra_doc_links
https://doc.rust-lang.org/rustdoc/lints.html
2021-08-11 19:48:51 +00:00
f5c63b2ae6
chore: update nom to version 6
...
Triggered some changes from `Fn` to `FnMut`.
2021-07-13 15:28:42 +02:00
d35b74c226
fix: Fix doc build warnings ( #1945 )
...
* fix: Fix doc build warnings
* refactor: add deny bare_urls to crates
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-13 08:03:42 +00:00
e6d995cbd8
chore: Update to Rust 1.53.0 ( #1922 )
...
* chore: Update to Rust 1.53.0
* fix: Update to latest clippy standards
* fix: bad refactor
* fix: Update escaping
* test: update test output
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-07 18:02:03 +00:00
eddc9319ff
docs: deny broken intradoc links
2021-04-27 13:22:28 +02:00
ceb233d6ec
refactor: satisfy new clippy lints
2021-04-20 17:30:50 +00:00
6e78dbbf07
fix: Remove manual reimplementations of map
2021-04-19 08:45:41 -04:00
82588d5c72
fix: Don't return Result from test functions
2021-04-07 12:40:00 -04:00
80d55d0829
chore: rename tracing_deps to observability_deps
...
OpenTelemetry makes this necessary.
2021-04-02 13:14:30 -07:00
0b880d3534
chore: Group all tracing-related crates under one crate for easier upgrade management
2021-04-02 09:54:39 -04:00
dd3a6a2d2e
refactor: make a tuple for 4 choices
2021-03-25 13:20:32 -04:00
5dfa84f0bb
fix: add e- and E-
2021-03-25 12:48:16 -04:00
31389f88df
refactor: use alt to reduce code line
2021-03-25 11:02:08 -04:00
ac480d8d8f
fix: support upper case for scientific notation
2021-03-25 10:14:31 -04:00
b758804df7
feat: add functions to suport scientific notations
2021-03-23 18:09:12 -04:00
13d361e645
feat: support sicientific float data type
2021-03-23 15:35:00 -04:00
255589cfcb
fix: test error type
2021-03-09 07:29:01 -08:00
abb991ffca
fix: add tests to unsigned integer parsing in LP and MUB
2021-03-08 15:29:05 -08:00
ac1b0c04ae
fix(line-protocol): add unsigned integer field type
...
Fixes #904
The line protocol parser was lacking the unsigned integer type, which
suffixes values with `u`. This adds unsigned integer support to the line
protocol parser, and fills a few corresponding gaps in the mutable
buffer.
2021-03-08 09:59:12 -08:00
dfe6e50e26
refactor: add arc clone lint
2021-02-15 12:52:12 +00:00
6f473984d0
style: wrap comments
...
Runs rustfmt with the new config.
2020-12-11 18:22:26 +00:00
Andrew Lamb
Dom Dwyer
Carol (Nichols || Goulding)
Marko Mikulicic
Marco Neumann
Nga Tran
Andrew Lamb
Edd Robinson
Carol (Nichols || Goulding)
Jacob Marble
Jacob Marble