How to import more than 2k lines of .csv file

-1

I really need a help. Here is a situation. Every monday I receive a .csv file on my Gmail Accout. Usually it is >3k rows and contains basically numeric data.

Up to now I was downloading this .csv file and copy/paste to the Google Sheets. Now I want to automate this task.

I have searched internet and found this pretty Script for Google Sheets:

function getCSV() {

  // Abrir a planhilha e limpar ela

  var sheetID = SpreadsheetApp.getActiveSpreadsheet().getId();

var ss = SpreadsheetApp.getActiveSpreadsheet();

 var sheet = SpreadsheetApp.openById(sheetID);

  var first = ss.getSheetByName("Agrupados_por_plano");

  first.clearContents();

  // 



 var myLabel = GmailApp.getUserLabelByName("Pulse MetaBase"); // specify label in gmail

 var threads = myLabel.getThreads(0,1);

 var msgs = GmailApp.getMessagesForThreads(threads);

 var attachments = msgs[0][0].getAttachments();

  //var content = attachments.getContentType();

  Logger.log(attachments[0].getSize());

 // 



 var csv =  attachments[0].getBlob().getDataAsString();





 var data = Utilities.parseCsv(csv);



var a = data.length ;

var b = data[0].length; 





  var range = first.getRange(1,1,data.length,data[0].length);

range.setValues(data);







}

The script does work pretty good and I have no errors. But it imports only first 2000 rows of data. For example, I have a file with 2780 rows and 5 columns. The script I use only imports 2000 rows and 5 columns and original file size (250 kBytes) is reduced to 192 kBytes.

How would I solve it? Is it a limitation of getAttachment() method?

edited Nov 21 at 1:38

Rubén

10k43265

asked Nov 21 at 0:17

andrii_ufu

3

In order to understand correctly your situation, can you provide the detail information about does not import more than 2k lines of .csv file? For example, those are the error messages, the size of CSV file and the values of CSV when it is not imported. I think that they will help users think of your solution.
– Tanaike
Nov 21 at 0:24

Hey, thank you for your comment. Actually script does import data, but only amout limited to 2000 rows.
– andrii_ufu
Nov 21 at 0:39

Thank you for replying and updating your question. In my environment, when I tried to import a CSV data with 3000 rows, 5 columns and 400 kb in the file size, all CSV data can be imported. For example, when the CSV data is imported to new Spreadsheet, does the same issue occur? If it's so, how about confirming the format of CSV data you want to import? I'm sorry I cannot propose the concrete method because I cannot replicate your situation. If this was not the direct solution of your issue, I'm sorry.
– Tanaike
Nov 21 at 1:04

1

What is the line separator used by your CSV file?
– Rubén
Nov 21 at 1:42

add a comment |

-1

I really need a help. Here is a situation. Every monday I receive a .csv file on my Gmail Accout. Usually it is >3k rows and contains basically numeric data.

Up to now I was downloading this .csv file and copy/paste to the Google Sheets. Now I want to automate this task.

I have searched internet and found this pretty Script for Google Sheets:

function getCSV() {

  // Abrir a planhilha e limpar ela

  var sheetID = SpreadsheetApp.getActiveSpreadsheet().getId();

var ss = SpreadsheetApp.getActiveSpreadsheet();

 var sheet = SpreadsheetApp.openById(sheetID);

  var first = ss.getSheetByName("Agrupados_por_plano");

  first.clearContents();

  // 



 var myLabel = GmailApp.getUserLabelByName("Pulse MetaBase"); // specify label in gmail

 var threads = myLabel.getThreads(0,1);

 var msgs = GmailApp.getMessagesForThreads(threads);

 var attachments = msgs[0][0].getAttachments();

  //var content = attachments.getContentType();

  Logger.log(attachments[0].getSize());

 // 



 var csv =  attachments[0].getBlob().getDataAsString();





 var data = Utilities.parseCsv(csv);



var a = data.length ;

var b = data[0].length; 





  var range = first.getRange(1,1,data.length,data[0].length);

range.setValues(data);







}

How would I solve it? Is it a limitation of getAttachment() method?

edited Nov 21 at 1:38

Rubén

10k43265

asked Nov 21 at 0:17

andrii_ufu

3

In order to understand correctly your situation, can you provide the detail information about does not import more than 2k lines of .csv file? For example, those are the error messages, the size of CSV file and the values of CSV when it is not imported. I think that they will help users think of your solution.
– Tanaike
Nov 21 at 0:24

Hey, thank you for your comment. Actually script does import data, but only amout limited to 2000 rows.
– andrii_ufu
Nov 21 at 0:39

Thank you for replying and updating your question. In my environment, when I tried to import a CSV data with 3000 rows, 5 columns and 400 kb in the file size, all CSV data can be imported. For example, when the CSV data is imported to new Spreadsheet, does the same issue occur? If it's so, how about confirming the format of CSV data you want to import? I'm sorry I cannot propose the concrete method because I cannot replicate your situation. If this was not the direct solution of your issue, I'm sorry.
– Tanaike
Nov 21 at 1:04

1

What is the line separator used by your CSV file?
– Rubén
Nov 21 at 1:42

add a comment |

-1

I really need a help. Here is a situation. Every monday I receive a .csv file on my Gmail Accout. Usually it is >3k rows and contains basically numeric data.

Up to now I was downloading this .csv file and copy/paste to the Google Sheets. Now I want to automate this task.

I have searched internet and found this pretty Script for Google Sheets:

function getCSV() {

  // Abrir a planhilha e limpar ela

  var sheetID = SpreadsheetApp.getActiveSpreadsheet().getId();

var ss = SpreadsheetApp.getActiveSpreadsheet();

 var sheet = SpreadsheetApp.openById(sheetID);

  var first = ss.getSheetByName("Agrupados_por_plano");

  first.clearContents();

  // 



 var myLabel = GmailApp.getUserLabelByName("Pulse MetaBase"); // specify label in gmail

 var threads = myLabel.getThreads(0,1);

 var msgs = GmailApp.getMessagesForThreads(threads);

 var attachments = msgs[0][0].getAttachments();

  //var content = attachments.getContentType();

  Logger.log(attachments[0].getSize());

 // 



 var csv =  attachments[0].getBlob().getDataAsString();





 var data = Utilities.parseCsv(csv);



var a = data.length ;

var b = data[0].length; 





  var range = first.getRange(1,1,data.length,data[0].length);

range.setValues(data);







}

How would I solve it? Is it a limitation of getAttachment() method?

edited Nov 21 at 1:38

Rubén

10k43265

asked Nov 21 at 0:17

andrii_ufu

I really need a help. Here is a situation. Every monday I receive a .csv file on my Gmail Accout. Usually it is >3k rows and contains basically numeric data.

Up to now I was downloading this .csv file and copy/paste to the Google Sheets. Now I want to automate this task.

I have searched internet and found this pretty Script for Google Sheets:

function getCSV() {

  // Abrir a planhilha e limpar ela

  var sheetID = SpreadsheetApp.getActiveSpreadsheet().getId();

var ss = SpreadsheetApp.getActiveSpreadsheet();

 var sheet = SpreadsheetApp.openById(sheetID);

  var first = ss.getSheetByName("Agrupados_por_plano");

  first.clearContents();

  // 



 var myLabel = GmailApp.getUserLabelByName("Pulse MetaBase"); // specify label in gmail

 var threads = myLabel.getThreads(0,1);

 var msgs = GmailApp.getMessagesForThreads(threads);

 var attachments = msgs[0][0].getAttachments();

  //var content = attachments.getContentType();

  Logger.log(attachments[0].getSize());

 // 



 var csv =  attachments[0].getBlob().getDataAsString();





 var data = Utilities.parseCsv(csv);



var a = data.length ;

var b = data[0].length; 





  var range = first.getRange(1,1,data.length,data[0].length);

range.setValues(data);







}

How would I solve it? Is it a limitation of getAttachment() method?

google-apps-script google-sheets

edited Nov 21 at 1:38

Rubén

10k43265

asked Nov 21 at 0:17

andrii_ufu

edited Nov 21 at 1:38

Rubén

10k43265

asked Nov 21 at 0:17

andrii_ufu

edited Nov 21 at 1:38

Rubén

10k43265

edited Nov 21 at 1:38

Rubén

10k43265

edited Nov 21 at 1:38

Rubén

10k43265

asked Nov 21 at 0:17

andrii_ufu

asked Nov 21 at 0:17

andrii_ufu

asked Nov 21 at 0:17

andrii_ufu

3

In order to understand correctly your situation, can you provide the detail information about does not import more than 2k lines of .csv file? For example, those are the error messages, the size of CSV file and the values of CSV when it is not imported. I think that they will help users think of your solution.
– Tanaike
Nov 21 at 0:24

Hey, thank you for your comment. Actually script does import data, but only amout limited to 2000 rows.
– andrii_ufu
Nov 21 at 0:39

Thank you for replying and updating your question. In my environment, when I tried to import a CSV data with 3000 rows, 5 columns and 400 kb in the file size, all CSV data can be imported. For example, when the CSV data is imported to new Spreadsheet, does the same issue occur? If it's so, how about confirming the format of CSV data you want to import? I'm sorry I cannot propose the concrete method because I cannot replicate your situation. If this was not the direct solution of your issue, I'm sorry.
– Tanaike
Nov 21 at 1:04

1

What is the line separator used by your CSV file?
– Rubén
Nov 21 at 1:42

add a comment |

3

In order to understand correctly your situation, can you provide the detail information about does not import more than 2k lines of .csv file? For example, those are the error messages, the size of CSV file and the values of CSV when it is not imported. I think that they will help users think of your solution.
– Tanaike
Nov 21 at 0:24

Hey, thank you for your comment. Actually script does import data, but only amout limited to 2000 rows.
– andrii_ufu
Nov 21 at 0:39

Thank you for replying and updating your question. In my environment, when I tried to import a CSV data with 3000 rows, 5 columns and 400 kb in the file size, all CSV data can be imported. For example, when the CSV data is imported to new Spreadsheet, does the same issue occur? If it's so, how about confirming the format of CSV data you want to import? I'm sorry I cannot propose the concrete method because I cannot replicate your situation. If this was not the direct solution of your issue, I'm sorry.
– Tanaike
Nov 21 at 1:04

1

What is the line separator used by your CSV file?
– Rubén
Nov 21 at 1:42

In order to understand correctly your situation, can you provide the detail information about does not import more than 2k lines of .csv file? For example, those are the error messages, the size of CSV file and the values of CSV when it is not imported. I think that they will help users think of your solution.
– Tanaike
Nov 21 at 0:24

Hey, thank you for your comment. Actually script does import data, but only amout limited to 2000 rows.
– andrii_ufu
Nov 21 at 0:39

Thank you for replying and updating your question. In my environment, when I tried to import a CSV data with 3000 rows, 5 columns and 400 kb in the file size, all CSV data can be imported. For example, when the CSV data is imported to new Spreadsheet, does the same issue occur? If it's so, how about confirming the format of CSV data you want to import? I'm sorry I cannot propose the concrete method because I cannot replicate your situation. If this was not the direct solution of your issue, I'm sorry.
– Tanaike
Nov 21 at 1:04

What is the line separator used by your CSV file?
– Rubén
Nov 21 at 1:42

add a comment |

1 Answer
1

active

oldest

votes

I found out what was the issue. It is all about paying attention to the code you are working on. My script was getting the same message over and over and that particular message contains .csv attach with exactly 2000 rows of data. Só what did I do? Just add few lines of code in order to get the last message in the thread every time my script is triggered.

function getThreadsGmail()

{

  // pegar o último email do thread e retornar arquivo anexo

  var myLabel = GmailApp.getUserLabelByName("Meta_Base_Reports"); // especificar lable em Gmail

 var threads = myLabel.getThreads();

 

  

   var msgs = threads[0].getMessages();

  var lastMsg = msgs[msgs.length - 1]; // get the last message in the thread

  var attachments = lastMsg.getAttachments();

  return attachments;

}

Thank you guys! =)

answered Nov 22 at 17:40

andrii_ufu

add a comment |

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53403549%2fhow-to-import-more-than-2k-lines-of-csv-file%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

function getThreadsGmail()

{

  // pegar o último email do thread e retornar arquivo anexo

  var myLabel = GmailApp.getUserLabelByName("Meta_Base_Reports"); // especificar lable em Gmail

 var threads = myLabel.getThreads();

 

  

   var msgs = threads[0].getMessages();

  var lastMsg = msgs[msgs.length - 1]; // get the last message in the thread

  var attachments = lastMsg.getAttachments();

  return attachments;

}

Thank you guys! =)

answered Nov 22 at 17:40

andrii_ufu

add a comment |

function getThreadsGmail()

{

  // pegar o último email do thread e retornar arquivo anexo

  var myLabel = GmailApp.getUserLabelByName("Meta_Base_Reports"); // especificar lable em Gmail

 var threads = myLabel.getThreads();

 

  

   var msgs = threads[0].getMessages();

  var lastMsg = msgs[msgs.length - 1]; // get the last message in the thread

  var attachments = lastMsg.getAttachments();

  return attachments;

}

Thank you guys! =)

answered Nov 22 at 17:40

andrii_ufu

add a comment |

function getThreadsGmail()

{

  // pegar o último email do thread e retornar arquivo anexo

  var myLabel = GmailApp.getUserLabelByName("Meta_Base_Reports"); // especificar lable em Gmail

 var threads = myLabel.getThreads();

 

  

   var msgs = threads[0].getMessages();

  var lastMsg = msgs[msgs.length - 1]; // get the last message in the thread

  var attachments = lastMsg.getAttachments();

  return attachments;

}

Thank you guys! =)

answered Nov 22 at 17:40

andrii_ufu

function getThreadsGmail()

{

  // pegar o último email do thread e retornar arquivo anexo

  var myLabel = GmailApp.getUserLabelByName("Meta_Base_Reports"); // especificar lable em Gmail

 var threads = myLabel.getThreads();

 

  

   var msgs = threads[0].getMessages();

  var lastMsg = msgs[msgs.length - 1]; // get the last message in the thread

  var attachments = lastMsg.getAttachments();

  return attachments;

}

Thank you guys! =)

function getThreadsGmail()

{

  // pegar o último email do thread e retornar arquivo anexo

  var myLabel = GmailApp.getUserLabelByName("Meta_Base_Reports"); // especificar lable em Gmail

 var threads = myLabel.getThreads();

 

  

   var msgs = threads[0].getMessages();

  var lastMsg = msgs[msgs.length - 1]; // get the last message in the thread

  var attachments = lastMsg.getAttachments();

  return attachments;

}

function getThreadsGmail()

{

  // pegar o último email do thread e retornar arquivo anexo

  var myLabel = GmailApp.getUserLabelByName("Meta_Base_Reports"); // especificar lable em Gmail

 var threads = myLabel.getThreads();

 

  

   var msgs = threads[0].getMessages();

  var lastMsg = msgs[msgs.length - 1]; // get the last message in the thread

  var attachments = lastMsg.getAttachments();

  return attachments;

}

answered Nov 22 at 17:40

andrii_ufu

answered Nov 22 at 17:40

andrii_ufu

answered Nov 22 at 17:40

andrii_ufu

answered Nov 22 at 17:40

andrii_ufu

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

Some of your past answers have not been well-received, and you're in danger of being blocked from answering.

Please pay close attention to the following guidance:

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Tukukkk