I am trying to fetch 392 rows from a table and print them on pdf. The pdf gets generated successfully but the records are not fetched correctly. Some records are missing on the pdf. I have observed every 22nd row is missing for some reason. Hence 17 records are missing out 392 records. I have tried to do many things to solve this but unsuccessful. Any help would be greatly appreciated.
We would need to see all the code involved here to be able to provide any assistance at all. You may also want to post an image of one of the pages of the PDF.
When posting code here, please copy/paste the code into the body of your post, surrounded between lines of three backtick - ` characters. This means you’ll have a line of ```, then your code, then another line of ```. This forces the forum software to keep your code properly formatted, which is critical with Python. Please do not post images of code.
However, in this case, it may end up being beneficial if you were to post an image of a page of the pdf.
I am posting the same issue with diff account as I was not able to login to my original account. I have attached the image of the pdf. There are total 392 records and only 365 are fetched. Every 22nd rows is missing. Hence 17 rows are missing. I tried creating a csv and it works fine. All the records are fetched in the csv file.
Following is the code from views.py and bill_dtls_data is the array where the rows are fetched before rendering.
def GeneratePdf_all(request):
user = request.session.get('w2_user')
file_date = request.session.get('file_date')
area = '11'
user_type = request.session.get('dept')
if user is not None:
bill_dt = request.GET.get('bill_dt')
bill_dt = bill_dt.split('-')
month_number=bill_dt[1]
month = calendar.month_name[int(month_number)]
bill_dt = bill_dt[2] + '-' + month + '-' + bill_dt[0]
orcl = area_wise_conn(area)
query = ("select to_char(bill_id) from mvendorbill where PERIOD_FROM=" + "'" + bill_dt + "'")
bill_dtls_data = []
table_data = db_select_query(query, area)
for i in table_data:
bill_id=i[0]
query = ("select bill_id,VENDOR_ID,name, to_char(PERIOD_FROM,'dd-mon-yyyy') PERIOD_FROM,to_char(PERIOD_TO,'dd-mon-yyyy') PERIOD_TO,BILL_AMT,PROCESSED_DT,PERIOD_FROM rundate,decode(VENDOR_ID,1,1,25,25,0) printcode from mvendorbill b,mpartner p where b.VENDOR_ID=p.VENDOR_CODE and bill_id=" + "" + bill_id + "")
bill_summary = []
table_data = db_select_query(query, area)
for i in table_data:
last_day_of_prev_month = i[7].replace(day=1) - timedelta(days=1)
payment_month=last_day_of_prev_month.strftime("%B" '-' "%Y")
bill_summary.append({'col1': i[0], 'col2': i[1], 'col3': i[2], 'col4': i[3], 'col5': i[4], 'col6': i[5], 'col7': i[6], 'col8': payment_month, 'col9': i[7]})
vendor_id_print=str(i[1])
bill_id_print=str(i[0])
query = ("select subs_no,tel_no,bill_no,to_char(bill_dt,'dd-MON-yy') bill_dt,ont_dep,rent_voice,usage_voice,rent_data,rent_cpe,to_char(pay_dt,'dd-MON-yy') pay_dt,pay_amnt,INSTRUMNT_CHRG,VOIC_ACTVTION,ONT_ACTVTION,ONT_OWNER,EXCHNG_CODE,sub_code,nvl(RENT_REBATE_DEL,0) RENT_REBATE_DEL,nvl(RENT_REBATE_TB,0) RENT_REBATE_TB,nvl(PON_WIRING,0) PON_WIRING,nvl(SALES_INCENTIVE,0) SALES_INCENTIVE from vendor_paid_processed where bill_id=" + "" + bill_id + " order by sub_code,pay_dt")
bill_dtls_data = []
table_data = db_select_query(query, area)
for i in table_data:
bill_dtls_data.append({'subs_no': i[0], 'tel_no': i[1], 'bill_no': i[2], 'bill_dt': i[3], 'ont_dep': i[4], 'rent_voice': i[5], 'usage_voice': i[6], 'rent_data': i[7], 'rent_cpe': i[8], 'pay_dt': i[9], 'pay_amnt': i[10], 'INSTRUMNT_CHRG': i[11], 'VOIC_ACTVTION': i[12], 'ONT_ACTVTION': i[13], 'ONT_OWNER': i[14], 'EXCHNG_CODE': i[15] , 'SUB_CODE': i[16], 'RENT_REBATE_DEL': i[17] , 'RENT_REBATE_TB': i[18],'PON_WIRING':i[19],'SALES_INCENTIVE':i[20] })
query = ("select sum(ont_dep) ont_dep,sum(rent_voice) rent_voice,sum(usage_voice) usage_voice,sum(rent_data) rent_data,sum(rent_cpe) rent_cpe,sum(pay_amnt) pay_amnt,sum(INSTRUMNT_CHRG) INSTRUMNT_CHRG,sum(VOIC_ACTVTION) VOIC_ACTVTION,sum(ONT_ACTVTION) ONT_ACTVTION ,count(*) total_count,SUM(nvl(RENT_REBATE_DEL,0)) RENT_REBATE_DEL,sum(nvl(RENT_REBATE_TB,0)) RENT_REBATE_TB ,sum(nvl(PON_WIRING,0)) PON_WIRING ,sum(nvl(SALES_INCENTIVE,0)) SALES_INCENTIVE from vendor_paid_processed where bill_id=" + "'" + bill_id + "'")
bill_dtls_data_summary = []
table_data = db_select_query(query, area)
for i in table_data:
bill_dtls_data_summary.append({'ont_dep': i[0], 'rent_voice': i[1], 'usage_voice': i[2], 'rent_data': i[3], 'rent_cpe': i[4], 'pay_amnt': i[5], 'INSTRUMNT_CHRG': i[6], 'VOIC_ACTVTION': i[7], 'ONT_ACTVTION': i[8], 'TOTAL_COUNT': i[9],'RENT_REBATE_DEL': i[10],'RENT_REBATE_TB': i[11],'PON_WIRING':i[12],'SALES_INCENTIVE':i[13] })
query = ("select distinct sub_code from mvendorbilldtl where bill_id=" + "'" + bill_id + "' order by sub_code")
table_data = db_select_query(query, area)
bill_dtls_all_newpolicy = []
type0=""
type1=""
type2=""
type3=""
bill_dtls_all = []
bill_dtls_all_1 = []
bill_dtls_all_2 = []
bill_dtls_all_3 = []
bill_dtls_csms = []
bill_dtls_csms_1 = []
bill_dtls_csms_2 = []
bill_dtls_csms_3 = []
for i in table_data:
sub_code=i[0]
cur = orcl.cursor()
p_bill_id = cur.var(cx_Oracle.NUMBER)
p_sub_code= cur.var(cx_Oracle.NUMBER)
oerrorcode = cur.var(cx_Oracle.STRING)
oerrormsg = cur.var(cx_Oracle.STRING)
curr = cur.var(cx_Oracle.CURSOR)
p_bill_id=bill_id
p_sub_code=sub_code
print(sub_code,bill_id)
if sub_code=="0" or sub_code=="1" or sub_code=="2" or sub_code=="3":
query = ("select sum(nvl(csms,0)) csms,sum(nvl(OLT,0)) OLT,sum(nvl(ONT,0)) ONT from (select decode(service_type,8,DTL_AMT) csms,decode(service_type,9,DTL_AMT) OLT,decode(service_type,22,DTL_AMT) ONT from mvendorbilldtl where service_type in (8,9,22) and sub_code=" + "'" + sub_code + "' and bill_id=" + "'" + bill_id + "')")
else:
query = ("select sum(nvl(csms,0)) csms,sum(nvl(OLT,0)) OLT,sum(nvl(ONT,0)) ONT from (select decode(service_type,8,DTL_AMT) csms,decode(service_type,9,DTL_AMT) OLT,decode(service_type,22,DTL_AMT) ONT from mvendorbilldtl where service_type in (8,9,22) and sub_code is null and bill_id=" + "'" + bill_id + "')")
#bill_dtls_csms = []
table_data = db_select_query(query, area)
for k in table_data:
if p_sub_code=="0":
type0="Y"
bill_dtls_csms.append({'csms': k[0], 'olt': k[1], 'ont': k[2]})
elif p_sub_code=="1":
type1="Y"
bill_dtls_csms_1.append({'csms': k[0], 'olt': k[1], 'ont': k[2]})
elif p_sub_code=="2":
type2="Y"
bill_dtls_csms_2.append({'csms': k[0], 'olt': k[1], 'ont': k[2]})
elif p_sub_code=="3":
type3="Y"
bill_dtls_csms_3.append({'csms': k[0], 'olt': k[1], 'ont': k[2]})
procedure3 = cur.callproc("WEIMS.all_bill_print_newpolicy",
[ curr,p_bill_id,p_sub_code,oerrormsg, oerrorcode])
for i in procedure3[0]:
if p_sub_code=="0":
if vendor_id_print=="25" and p_sub_code=="0":
bill_dtls_all.append({'col1': i[0], 'col2': i[1], 'col3': i[2], 'col4': i[3], 'col5': i[4], 'col6': i[5], 'col7': i[6], 'col8': i[7], 'col9': i[8]})
else:
bill_dtls_all.append({'col1': i[0], 'col2': i[1], 'col3': i[2], 'col4': i[3], 'col5': i[4], 'col6': i[5], 'col7': i[6]})
if p_sub_code=="1":
bill_dtls_all_1.append({'col1': i[0], 'col2': i[1], 'col3': i[2], 'col4': i[3], 'col5': i[4], 'col6': i[5], 'col7': i[6]})
if p_sub_code=="2":
bill_dtls_all_2.append({'col1': i[0], 'col2': i[1], 'col3': i[2], 'col4': i[3], 'col5': i[4], 'col6': i[5], 'col7': i[6]})
if p_sub_code=="3":
bill_dtls_all_3.append({'col1': i[0], 'col2': i[1], 'col3': i[2], 'col4': i[3], 'col5': i[4], 'col6': i[5], 'col7': i[6]})
print(bill_dtls_all)
print(bill_dtls_all_1)
print(type1)
data = json.dumps([bill_summary, bill_dtls_all, user_type], indent=4, sort_keys=True, default=str)
template = get_template("revenue_share/pdf.html")
content={'f':[[1,2,3,4],[4,5,6,7]]}
html = template.render({'bill_dtls_all':bill_dtls_all,'bill_dtls_all_1':bill_dtls_all_1,'bill_dtls_all_2':bill_dtls_all_2,'bill_dtls_all_3':bill_dtls_all_3,'bill_summary':bill_summary ,'bill_dtls_csms':bill_dtls_csms,'bill_dtls_data':bill_dtls_data,'bill_dtls_data_summary':bill_dtls_data_summary,'content':content,'type0':type0,'type1':type1,'type2':type2,'type3':type3,'bill_dtls_csms':bill_dtls_csms,'bill_dtls_csms_1':bill_dtls_csms_1,'bill_dtls_csms_2':bill_dtls_csms_2,'bill_dtls_csms_3':bill_dtls_csms_3}) # Renders the template with the context data.
options = {'enable-local-file-access': None,
'page-size': 'A3',
'margin-top': '0.75in',
'margin-right': '0.75in',
'margin-bottom': '0.75in',
'margin-left': '0.75in',}
file_name="E:\Revenue_share\FTTH_Bills\Bill_" + vendor_id_print + "_" + bill_id_print + "_" + file_date + ".pdf"
pdfkit.from_string(html,file_name , options=options)
pdf = open(file_name, encoding="mbcs")
with open(file_name, 'rb') as file:
binaryData = file.read()
curs = orcl.cursor()
try:
curs.execute("update mvendorbill set bill_pdf=:1 where PERIOD_FROM=:2 and BILL_ID=:3",
[binaryData,bill_dt, bill_id])
message = 'success'
except Exception as e:
message = e
if message == 'success':
orcl.commit()
return render(request, 'revenue_share/login.html')
Is this a case where there’s a row being dropped off of each page? (Are there 21 rows being rendered on a page? What happens if you change the dimensions or margins on the page?)
Or is there something different about the 22nd row from the 21 rows before it? (A summary or subtotal or something along those lines?)
What makes this number (21 or 22) special or different in the data you are processing?
Thank you for the quick response KenWhitesell
Is this a case where there’s a row being dropped off of each page? (Are there 21 rows being rendered on a page? What happens if you change the dimensions or margins on the page?)
→ 31 rows are rendered on each page. We tried changing the dimensions and it did not help.
Or is there something different about the 22nd row from the 21 rows before it? (A summary or subtotal or something along those lines?)
→ I compared the data between the missing rows and the rows displayed but I don’t see anything different in the missing rows.
What makes this number (21 or 22) special or different in the data you are processing?
→ We are trying to figure that out too.
In addition, I tried to append the missing rows at the end of the bill_dtls_data. It displayed the rows at the end but the same issue occurs again.
It is again skipping the row which comes on 22nd line pattern. We think, this is an issue with pdfkit library but not sure about it. If this can be resolved or if there is a work around for this, it will be really helpful.
It looks like you’re rendering the entire document as a string and then writing that string to a pdf.
Have you tried writing html to a file and visually verifying that that is correct? (I think the next thing to check would be the template.)
Hi KenWhitesell, thank you for the solution.
I used pdfkit.from_file instead of pdfkit.from_string and it is working file.
Thank you so much!!!
Hi KenWhitesell,
After using pdfkit.from_file , it worked only once and again we are facing the same issue. Could you please help?
I don’t have any specific ideas or knowledge of this, I was just trying to suggest ways of finding where things are going wrong. If you’re writing the rendered template to a file, I’d suggest examining that file to see if it looks right.
The html file display correct data but the pdf has missing data.
And you can’t see anything different, unusual, or questionable about those rows that are being dropped?
Don’t just examine the data, also look at the structure of the data file. Look for unprintable characters or some other anomaly that might explain this.