How to permanently record/update few characters worth of stats, python
I'm trying to record some stats on a script I'm running in python (a few percentages, less than 12 characters worth). I want it to be efficient. I want the stats to keep being updated as the script runs so that if the script were to exit I still have the stats available to be updated when I run the script again.
I've thought of ways such as recording in a csv (seems inefficient since there looks to be no functionality to keep updating the same row ), updating the title of a file within the system. But can think of nothing which is as clean and efficient as I was hoping. Any ideas?
python
asked Nov 22 '18 at 2:40
Finn MaunsellFinn Maunsell
388
add a comment |
I'm trying to record some stats on a script I'm running in python (a few percentages, less than 12 characters worth). I want it to be efficient. I want the stats to keep being updated as the script runs so that if the script were to exit I still have the stats available to be updated when I run the script again.
I've thought of ways such as recording in a csv (seems inefficient since there looks to be no functionality to keep updating the same row ), updating the title of a file within the system. But can think of nothing which is as clean and efficient as I was hoping. Any ideas?
python
asked Nov 22 '18 at 2:40
Finn MaunsellFinn Maunsell
388
1
For small scripts, with not many updates write, it's fine to store to a simple txt file, you don't need csv with just a single value. But if there are many operations, the best way might be to use SQLite, and sqlite3 as driver in Python.
– hunzter
Nov 22 '18 at 2:59
add a comment |
I'm trying to record some stats on a script I'm running in python (a few percentages, less than 12 characters worth). I want it to be efficient. I want the stats to keep being updated as the script runs so that if the script were to exit I still have the stats available to be updated when I run the script again.
I've thought of ways such as recording in a csv (seems inefficient since there looks to be no functionality to keep updating the same row ), updating the title of a file within the system. But can think of nothing which is as clean and efficient as I was hoping. Any ideas?
python
asked Nov 22 '18 at 2:40
Finn MaunsellFinn Maunsell
388
I'm trying to record some stats on a script I'm running in python (a few percentages, less than 12 characters worth). I want it to be efficient. I want the stats to keep being updated as the script runs so that if the script were to exit I still have the stats available to be updated when I run the script again.
I've thought of ways such as recording in a csv (seems inefficient since there looks to be no functionality to keep updating the same row ), updating the title of a file within the system. But can think of nothing which is as clean and efficient as I was hoping. Any ideas?
python
python
asked Nov 22 '18 at 2:40
Finn MaunsellFinn Maunsell
388
asked Nov 22 '18 at 2:40
Finn MaunsellFinn Maunsell
388
asked Nov 22 '18 at 2:40
Finn MaunsellFinn Maunsell
388
asked Nov 22 '18 at 2:40
Finn MaunsellFinn Maunsell
388
asked Nov 22 '18 at 2:40
Finn MaunsellFinn Maunsell
388
388
1
For small scripts, with not many updates write, it's fine to store to a simple txt file, you don't need csv with just a single value. But if there are many operations, the best way might be to use SQLite, and sqlite3 as driver in Python.
– hunzter
Nov 22 '18 at 2:59
add a comment |
1
For small scripts, with not many updates write, it's fine to store to a simple txt file, you don't need csv with just a single value. But if there are many operations, the best way might be to use SQLite, and sqlite3 as driver in Python.
– hunzter
Nov 22 '18 at 2:59
1
1
For small scripts, with not many updates write, it's fine to store to a simple txt file, you don't need csv with just a single value. But if there are many operations, the best way might be to use SQLite, and sqlite3 as driver in Python.
– hunzter
Nov 22 '18 at 2:59
For small scripts, with not many updates write, it's fine to store to a simple txt file, you don't need csv with just a single value. But if there are many operations, the best way might be to use SQLite, and sqlite3 as driver in Python.
– hunzter
Nov 22 '18 at 2:59
add a comment |
1 Answer
1
active
oldest
votes
You could store it in a .txt file if you'd like really, or anything. Using the built in module for .csv files in Python, overwriting rows directly - rather than having to recreate the file - isn't possible AFAIK. Check out the sqlite3 module for storing the information in a database.
https://docs.python.org/3/library/sqlite3.html
answered Nov 22 '18 at 2:49
Lucas LeodlerLucas Leodler
534
Good ideas. Any idea on the efficiency of using an sql database for this vs. opening a .txt file recording the text and closing the file?
– Finn Maunsell
Nov 22 '18 at 2:56
You could do this perfectly fine on text file if that's okay with you when you are reading it. When you are creating a solution remember to manipulate all of your data within python in some sort of a data structure before making a system "write()" call. Without implementing this standard your performance would decrease incrementally as you scaled the amount of data points per epoch. How often do you play on performing these operations and how long do you want to store the data? Do you plan on manipulating it in the future? These are three questions that could all lead to different answers.
– Lucas Leodler
Nov 22 '18 at 3:14
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53423141%2fhow-to-permanently-record-update-few-characters-worth-of-stats-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
You could store it in a .txt file if you'd like really, or anything. Using the built in module for .csv files in Python, overwriting rows directly - rather than having to recreate the file - isn't possible AFAIK. Check out the sqlite3 module for storing the information in a database.
https://docs.python.org/3/library/sqlite3.html
answered Nov 22 '18 at 2:49
Lucas LeodlerLucas Leodler
534
Good ideas. Any idea on the efficiency of using an sql database for this vs. opening a .txt file recording the text and closing the file?
– Finn Maunsell
Nov 22 '18 at 2:56
You could do this perfectly fine on text file if that's okay with you when you are reading it. When you are creating a solution remember to manipulate all of your data within python in some sort of a data structure before making a system "write()" call. Without implementing this standard your performance would decrease incrementally as you scaled the amount of data points per epoch. How often do you play on performing these operations and how long do you want to store the data? Do you plan on manipulating it in the future? These are three questions that could all lead to different answers.
– Lucas Leodler
Nov 22 '18 at 3:14
add a comment |
You could store it in a .txt file if you'd like really, or anything. Using the built in module for .csv files in Python, overwriting rows directly - rather than having to recreate the file - isn't possible AFAIK. Check out the sqlite3 module for storing the information in a database.
https://docs.python.org/3/library/sqlite3.html
answered Nov 22 '18 at 2:49
Lucas LeodlerLucas Leodler
534
Good ideas. Any idea on the efficiency of using an sql database for this vs. opening a .txt file recording the text and closing the file?
– Finn Maunsell
Nov 22 '18 at 2:56
You could do this perfectly fine on text file if that's okay with you when you are reading it. When you are creating a solution remember to manipulate all of your data within python in some sort of a data structure before making a system "write()" call. Without implementing this standard your performance would decrease incrementally as you scaled the amount of data points per epoch. How often do you play on performing these operations and how long do you want to store the data? Do you plan on manipulating it in the future? These are three questions that could all lead to different answers.
– Lucas Leodler
Nov 22 '18 at 3:14
add a comment |
You could store it in a .txt file if you'd like really, or anything. Using the built in module for .csv files in Python, overwriting rows directly - rather than having to recreate the file - isn't possible AFAIK. Check out the sqlite3 module for storing the information in a database.
https://docs.python.org/3/library/sqlite3.html
answered Nov 22 '18 at 2:49
Lucas LeodlerLucas Leodler
534
You could store it in a .txt file if you'd like really, or anything. Using the built in module for .csv files in Python, overwriting rows directly - rather than having to recreate the file - isn't possible AFAIK. Check out the sqlite3 module for storing the information in a database.
https://docs.python.org/3/library/sqlite3.html
answered Nov 22 '18 at 2:49
Lucas LeodlerLucas Leodler
534
answered Nov 22 '18 at 2:49
Lucas LeodlerLucas Leodler
534
answered Nov 22 '18 at 2:49
Lucas LeodlerLucas Leodler
534
answered Nov 22 '18 at 2:49
Lucas LeodlerLucas Leodler
534
534
Good ideas. Any idea on the efficiency of using an sql database for this vs. opening a .txt file recording the text and closing the file?
– Finn Maunsell
Nov 22 '18 at 2:56
You could do this perfectly fine on text file if that's okay with you when you are reading it. When you are creating a solution remember to manipulate all of your data within python in some sort of a data structure before making a system "write()" call. Without implementing this standard your performance would decrease incrementally as you scaled the amount of data points per epoch. How often do you play on performing these operations and how long do you want to store the data? Do you plan on manipulating it in the future? These are three questions that could all lead to different answers.
– Lucas Leodler
Nov 22 '18 at 3:14
add a comment |
Good ideas. Any idea on the efficiency of using an sql database for this vs. opening a .txt file recording the text and closing the file?
– Finn Maunsell
Nov 22 '18 at 2:56
You could do this perfectly fine on text file if that's okay with you when you are reading it. When you are creating a solution remember to manipulate all of your data within python in some sort of a data structure before making a system "write()" call. Without implementing this standard your performance would decrease incrementally as you scaled the amount of data points per epoch. How often do you play on performing these operations and how long do you want to store the data? Do you plan on manipulating it in the future? These are three questions that could all lead to different answers.
– Lucas Leodler
Nov 22 '18 at 3:14
Good ideas. Any idea on the efficiency of using an sql database for this vs. opening a .txt file recording the text and closing the file?
– Finn Maunsell
Nov 22 '18 at 2:56
Good ideas. Any idea on the efficiency of using an sql database for this vs. opening a .txt file recording the text and closing the file?
– Finn Maunsell
Nov 22 '18 at 2:56
You could do this perfectly fine on text file if that's okay with you when you are reading it. When you are creating a solution remember to manipulate all of your data within python in some sort of a data structure before making a system "write()" call. Without implementing this standard your performance would decrease incrementally as you scaled the amount of data points per epoch. How often do you play on performing these operations and how long do you want to store the data? Do you plan on manipulating it in the future? These are three questions that could all lead to different answers.
– Lucas Leodler
Nov 22 '18 at 3:14
You could do this perfectly fine on text file if that's okay with you when you are reading it. When you are creating a solution remember to manipulate all of your data within python in some sort of a data structure before making a system "write()" call. Without implementing this standard your performance would decrease incrementally as you scaled the amount of data points per epoch. How often do you play on performing these operations and how long do you want to store the data? Do you plan on manipulating it in the future? These are three questions that could all lead to different answers.
– Lucas Leodler
Nov 22 '18 at 3:14
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53423141%2fhow-to-permanently-record-update-few-characters-worth-of-stats-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
For small scripts, with not many updates write, it's fine to store to a simple txt file, you don't need csv with just a single value. But if there are many operations, the best way might be to use SQLite, and sqlite3 as driver in Python.
– hunzter
Nov 22 '18 at 2:59