Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
4,161 Commits
211 Branches
453 Tags
359 MiB
Commit Graph

4161 Commits (0fe8eda89e66c849e20c04f0fb90e4d3e4879663)

Author SHA1 Message Date
kodiakhq[bot] c8126784a8
Merge branch 'main' into ntran/avoid_sort_in_scan 2021-07-08 20:22:18 +00:00
Nga Tran 680394b50b refactor: run fmt 2021-07-08 16:21:42 -04:00
Andrew Lamb 72928aab3d
refactor: Move ChunkLifecycleAction to the data_types crate (#1939) 2021-07-08 20:18:33 +00:00
Nga Tran c5733ab4a7 refactor: remove redudant code 2021-07-08 16:11:42 -04:00
Nga Tran 6738cb272f refactor: remove duplicate test 2021-07-08 15:59:25 -04:00
Nga Tran da6249a4df fix: address reviewers' comments and also fixe a bug they discovered 2021-07-08 15:54:54 -04:00
Andrew Lamb dd3eff7748
refactor: Always use `row_count` for count of rows in system.* tables (#1937) 2021-07-08 19:28:11 +00:00
Andrew Lamb 8b9b369189
chore: Update DataFusion again (#1930) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-07-08 15:14:45 +00:00
Carol (Nichols || Goulding) c6bf0a26f4 feat: Add metrics for when ingesting from the write buffer fails 
So that we have some way of figuring out what might be going on.
 2021-07-08 09:57:51 -04:00
kodiakhq[bot] 9c722959fb
Merge pull request #1752 from influxdata/cn/kafka-read 
feat: reading entries from Kafka
 2021-07-08 13:39:16 +00:00
Carol (Nichols || Goulding) dd6303e85d test: Make test data conform to Kafka partitioning assumptions 2021-07-08 09:31:52 -04:00
Carol (Nichols || Goulding) 80e1dcafe0 feat: Support reading from all Kafka partitions 
When reading from the Kafka write buffer, subscribe to all partitions in
a topic and start from the smallest offset available, instead of
assuming there will only be 1 partition per topic.
 2021-07-08 09:30:59 -04:00
Carol (Nichols || Goulding) c90ef7b14b fix: Create one consumer group per server+database 
This hasn't caused any problems for me yet, but seemed like a good idea
because we want to be sure we don't get any of Kafka's consumer
rebalancing if we have multiple partitions.
 2021-07-08 09:28:34 -04:00
Carol (Nichols || Goulding) e5168936f5 feat: Better error messages through to gRPC API + e2e Kafka Read tests 2021-07-08 09:28:34 -04:00
Carol (Nichols || Goulding) c53ae41d57 fix: Remove unneeded Option from the reading mock 2021-07-08 09:28:34 -04:00
Carol (Nichols || Goulding) 854c28c41a feat: Stream messages from Kafka into the database 2021-07-08 09:28:34 -04:00
Carol (Nichols || Goulding) ee500f5bda feat: Support configuring a write buffer for writing OR reading 2021-07-08 09:28:34 -04:00
Carol (Nichols || Goulding) 63d26f6f3f refactor: Rename KafkaBuffer to KafkaBufferProducer 2021-07-08 09:28:34 -04:00
Carol (Nichols || Goulding) e5de73133c feat: Change write buffer connection rule to take either Writing or Reading connection info 
A database on one IOx server can, exclusively:

- Not interact with Kafka at all
- Send writes to Kafka
- Read writes from Kafka

Notably, a database on a particular server will never write *and* read from Kafka at the same time.
 2021-07-08 09:28:34 -04:00
Carol (Nichols || Goulding) fd4bcc2fa5 refactor: Rename the WriteBuffer trait to be WriteBufferWriting 2021-07-08 09:28:34 -04:00
Carol (Nichols || Goulding) 83e50cfba4 refactor: Rename field to not contain the type 2021-07-08 09:28:34 -04:00
kodiakhq[bot] 3a329d3705
Merge pull request #1931 from influxdata/crepererum/str_arcs 
refactor: convert some table name and part. key String to Arcs
 2021-07-08 13:28:04 +00:00
kodiakhq[bot] 69e4786fc7
Merge branch 'main' into crepererum/str_arcs 2021-07-08 13:20:49 +00:00
Marco Neumann 18893e76e0 refactor: convert some table name and part. key String to Arcs 
This has the (somewhat nice) side effect that it shrinks the in-mem
catalog a bit as well because nw `ParquetChunk` is a bit smaller making
the chunk stage enum smaller as well.
 2021-07-08 14:34:28 +02:00
kodiakhq[bot] 4c05d7ccfd
Merge pull request #1928 from influxdata/er/refactor/logging 
refactor: log information for compaction results
 2021-07-08 11:57:11 +00:00
Edd Robinson 7ff8ae4ce5 refactor: tidy up sort key rep 2021-07-08 12:48:41 +01:00
Edd Robinson f811bf1e5e refactor: log compaction activity 2021-07-08 12:48:41 +01:00
Edd Robinson 86372c8818 refactor: remove unused method 2021-07-08 12:48:41 +01:00
Andrew Lamb 33bc85ad18
feat: Infrastructure for persistence (#1925) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-07-08 11:14:38 +00:00
Andrew Lamb f670224ea1
chore: Reduce output spew during query tests (#1926) 
* chore: Reduce output spew during query tests

* docs: Update query_tests/src/runner.rs

Co-authored-by: Edd Robinson <me@edd.io>

Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-07-08 11:06:24 +00:00
Andrew Lamb 7602bde850
chore: Update datafusion deps (#1799) 
* chore: Update datafusion deps + rework code

* refactor: remove workaround as it has been contributed upstream

* fix: Update query/src/exec/split.rs

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-07-08 10:58:32 +00:00
kodiakhq[bot] 7b18c29d32
Merge pull request #1915 from influxdata/fixnonelinger 
fix: Ensure mutable_linger_seconds can not be None
 2021-07-08 09:33:15 +00:00
Marko Mikulicic 7059f16b9e
refactor: Turn mutable_linger_seconds into a non-optional (#1917) 2021-07-08 11:25:57 +02:00
Marko Mikulicic 0ec11eb907
fix: Handle None mutable_linger_seconds in can_move 
This will be followed by a cleanup PR that will no longer make mutable_linger_seconds an option
and move the defaulting to the configuration layer.

Closes #1899
 2021-07-08 11:02:09 +02:00
kodiakhq[bot] ea2bbab0f1
Merge pull request #1913 from influxdata/crepererum/issue1897 
feat: store schemas per table, enforce schema on write
 2021-07-08 08:08:59 +00:00
Marco Neumann 24056d7bfc test: ensure that table schemas are recovered from pres. catalog 2021-07-08 10:01:42 +02:00
Marco Neumann a746cd45c5 test: check for schema change errors 2021-07-08 09:51:49 +02:00
Marco Neumann bd22dd38ea docs: fix typos 
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
 2021-07-08 09:18:09 +02:00
Marco Neumann b528ac2b55 feat: store schemas per table 
This way we can:

- check for schema matches even for writes going into different
  partitions
- solve #1768 and #1884 in some future PR

Closes #1897.
 2021-07-08 09:18:09 +02:00
Marco Neumann 5ca9760c94 test: make partioning in DB tests consistent w/ DB rules 2021-07-08 09:18:09 +02:00
Marco Neumann ed3ebdcbd2 refactor: use sync locks w/ better metrics 2021-07-08 09:18:09 +02:00
Marco Neumann 5936452895 feat: add infra to check table-wide schemas 2021-07-08 09:18:09 +02:00
Marco Neumann 315217362f feat: add TableBatch -> Schema conversion 2021-07-08 09:18:09 +02:00
Nga Tran 5c722af0fa fix: remove comments 2021-07-07 16:50:53 -04:00
Nga Tran d3c4f8c249 fix: store sort key correctly inthe schema. Update tests to reflect it 2021-07-07 15:55:23 -04:00
kodiakhq[bot] 4e5d5c8c4c
Merge pull request #1924 from influxdata/pd-fix-last-write 
fix: don't update last write time on failed writes
 2021-07-07 19:07:37 +00:00
Paul Dix cc350bb1ea fix: don't update last write time on failed writes 
Fixes #1905
 2021-07-07 14:50:03 -04:00
kodiakhq[bot] ac1c7a5e07
Merge pull request #1923 from influxdata/cn/one-more-fb-fix 
fix: Outdated reference to a moved file in docs
 2021-07-07 18:11:47 +00:00
kodiakhq[bot] 94f32e357f
Merge branch 'main' into cn/one-more-fb-fix 2021-07-07 18:02:39 +00:00
Andrew Lamb e6d995cbd8
chore: Update to Rust 1.53.0 (#1922) 
* chore: Update to Rust 1.53.0

* fix: Update to latest clippy standards

* fix: bad refactor

* fix: Update escaping

* test: update test output

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-07-07 18:02:03 +00:00