Can arangodb handle 20 billion records?
up vote
1
down vote
favorite
I just want to know, can Arangodb handle 20 billions of records in a single collection?
What is the best way to insert 1 billion records at a time?
arangodb arangodb-php
asked Nov 19 at 12:20
Yash Pandya
61
New contributor
Yash Pandya is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
up vote
1
down vote
favorite
I just want to know, can Arangodb handle 20 billions of records in a single collection?
What is the best way to insert 1 billion records at a time?
arangodb arangodb-php
asked Nov 19 at 12:20
Yash Pandya
61
New contributor
Yash Pandya is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I just want to know, can Arangodb handle 20 billions of records in a single collection?
What is the best way to insert 1 billion records at a time?
arangodb arangodb-php
asked Nov 19 at 12:20
Yash Pandya
61
New contributor
Yash Pandya is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
I just want to know, can Arangodb handle 20 billions of records in a single collection?
What is the best way to insert 1 billion records at a time?
arangodb arangodb-php
arangodb arangodb-php
asked Nov 19 at 12:20
Yash Pandya
61
New contributor
Yash Pandya is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Nov 19 at 12:20
Yash Pandya
61
New contributor
Yash Pandya is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Nov 19 at 12:20
Yash Pandya
61
New contributor
Yash Pandya is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Nov 19 at 12:20
Yash Pandya
61
asked Nov 19 at 12:20
Yash Pandya
61
61
New contributor
Yash Pandya is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Yash Pandya is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Yash Pandya is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
Of course. The underlying storage engine choice should definitely be RocksDB. How big would every individual document in average be?
The best way would be any sort of import/restore. Is this going to be a cluster or a single server deployment?
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1401416
The individual document can be of size 1 kb. Currently I am using it as a single server and mmfiles as a storage engine.
– Yash Pandya
Nov 20 at 13:40
MMFiles is not an option unless you are looking into buying 20 exabyte of RAM. Memory mapped files performance goes towards zero, when data size including indexes cannot be matched in RAM size.
– Kaveh Vahedipour
Nov 20 at 17:45
Thanks for the reply. You suggested the best way to insert records is import/restore, but to use that my whole data should be ready before insertion,right? Suppose my data is generated at run time using some sort of script, at that time what should I use?
– Yash Pandya
yesterday
Absolutely. By all means insert all you data.
– Kaveh Vahedipour
yesterday
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
Of course. The underlying storage engine choice should definitely be RocksDB. How big would every individual document in average be?
The best way would be any sort of import/restore. Is this going to be a cluster or a single server deployment?
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1401416
The individual document can be of size 1 kb. Currently I am using it as a single server and mmfiles as a storage engine.
– Yash Pandya
Nov 20 at 13:40
MMFiles is not an option unless you are looking into buying 20 exabyte of RAM. Memory mapped files performance goes towards zero, when data size including indexes cannot be matched in RAM size.
– Kaveh Vahedipour
Nov 20 at 17:45
Thanks for the reply. You suggested the best way to insert records is import/restore, but to use that my whole data should be ready before insertion,right? Suppose my data is generated at run time using some sort of script, at that time what should I use?
– Yash Pandya
yesterday
Absolutely. By all means insert all you data.
– Kaveh Vahedipour
yesterday
add a comment |
up vote
0
down vote
Of course. The underlying storage engine choice should definitely be RocksDB. How big would every individual document in average be?
The best way would be any sort of import/restore. Is this going to be a cluster or a single server deployment?
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1401416
The individual document can be of size 1 kb. Currently I am using it as a single server and mmfiles as a storage engine.
– Yash Pandya
Nov 20 at 13:40
MMFiles is not an option unless you are looking into buying 20 exabyte of RAM. Memory mapped files performance goes towards zero, when data size including indexes cannot be matched in RAM size.
– Kaveh Vahedipour
Nov 20 at 17:45
Thanks for the reply. You suggested the best way to insert records is import/restore, but to use that my whole data should be ready before insertion,right? Suppose my data is generated at run time using some sort of script, at that time what should I use?
– Yash Pandya
yesterday
Absolutely. By all means insert all you data.
– Kaveh Vahedipour
yesterday
add a comment |
up vote
0
down vote
up vote
0
down vote
Of course. The underlying storage engine choice should definitely be RocksDB. How big would every individual document in average be?
The best way would be any sort of import/restore. Is this going to be a cluster or a single server deployment?
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1401416
Of course. The underlying storage engine choice should definitely be RocksDB. How big would every individual document in average be?
The best way would be any sort of import/restore. Is this going to be a cluster or a single server deployment?
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1401416
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1401416
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1401416
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1401416
2,1401416
The individual document can be of size 1 kb. Currently I am using it as a single server and mmfiles as a storage engine.
– Yash Pandya
Nov 20 at 13:40
MMFiles is not an option unless you are looking into buying 20 exabyte of RAM. Memory mapped files performance goes towards zero, when data size including indexes cannot be matched in RAM size.
– Kaveh Vahedipour
Nov 20 at 17:45
Thanks for the reply. You suggested the best way to insert records is import/restore, but to use that my whole data should be ready before insertion,right? Suppose my data is generated at run time using some sort of script, at that time what should I use?
– Yash Pandya
yesterday
Absolutely. By all means insert all you data.
– Kaveh Vahedipour
yesterday
add a comment |
The individual document can be of size 1 kb. Currently I am using it as a single server and mmfiles as a storage engine.
– Yash Pandya
Nov 20 at 13:40
MMFiles is not an option unless you are looking into buying 20 exabyte of RAM. Memory mapped files performance goes towards zero, when data size including indexes cannot be matched in RAM size.
– Kaveh Vahedipour
Nov 20 at 17:45
Thanks for the reply. You suggested the best way to insert records is import/restore, but to use that my whole data should be ready before insertion,right? Suppose my data is generated at run time using some sort of script, at that time what should I use?
– Yash Pandya
yesterday
Absolutely. By all means insert all you data.
– Kaveh Vahedipour
yesterday
The individual document can be of size 1 kb. Currently I am using it as a single server and mmfiles as a storage engine.
– Yash Pandya
Nov 20 at 13:40
The individual document can be of size 1 kb. Currently I am using it as a single server and mmfiles as a storage engine.
– Yash Pandya
Nov 20 at 13:40
MMFiles is not an option unless you are looking into buying 20 exabyte of RAM. Memory mapped files performance goes towards zero, when data size including indexes cannot be matched in RAM size.
– Kaveh Vahedipour
Nov 20 at 17:45
MMFiles is not an option unless you are looking into buying 20 exabyte of RAM. Memory mapped files performance goes towards zero, when data size including indexes cannot be matched in RAM size.
– Kaveh Vahedipour
Nov 20 at 17:45
Thanks for the reply. You suggested the best way to insert records is import/restore, but to use that my whole data should be ready before insertion,right? Suppose my data is generated at run time using some sort of script, at that time what should I use?
– Yash Pandya
yesterday
Thanks for the reply. You suggested the best way to insert records is import/restore, but to use that my whole data should be ready before insertion,right? Suppose my data is generated at run time using some sort of script, at that time what should I use?
– Yash Pandya
yesterday
Absolutely. By all means insert all you data.
– Kaveh Vahedipour
yesterday
Absolutely. By all means insert all you data.
– Kaveh Vahedipour
yesterday
add a comment |
Yash Pandya is a new contributor. Be nice, and check out our Code of Conduct.
draft saved
draft discarded
Yash Pandya is a new contributor. Be nice, and check out our Code of Conduct.
Yash Pandya is a new contributor. Be nice, and check out our Code of Conduct.
Yash Pandya is a new contributor. Be nice, and check out our Code of Conduct.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53374517%2fcan-arangodb-handle-20-billion-records%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
