59 replies to this topic

[dyan]

Hi GR,

 

I've tried the new code. Still unable to download. I'm on a Mac OS X Yosemite.

 

For the Expanded Bible (EXB), I get the following error:

Traceback (most recent call last):
  File "/Users/***/Downloads/bible_gateway1.py", line 569, in <module>
    translation_name = Copyright(translation).search(base_url)
  File "/Users/***/Downloads/bible_gateway1.py", line 156, in search
    self.publisher_info = soup.p.a.text
AttributeError: 'NoneType' object has no attribute 'a'

And for other bible, such as KJV, ESV

Traceback (most recent call last):
  File "/Users/***/Downloads/bible_gateway1.py", line 569, in <module>
    translation_name = Copyright(translation).search(base_url)
  File "/Users/***/Downloads/bible_gateway1.py", line 158, in search
    self.accept_me()
  File "/Users/***/Downloads/bible_gateway1.py", line 179, in accept_me
    ''.join((self.copyright_info_bak.strip())),
TypeError: sequence item 0: expected str instance, int found

[Aaron Viscichini]

Hi GreenRaccoon,

First of all, awesome job! The stand alone is amazing.

However, there is a strange bug I found when doing a bible study on 2 Timothy 3 using bibles created by your script. In 2 Timothy 3 and 4 there are verses copied from the end of the chapter which are inserted into the beginning of the chapter. I'm not sure why and I haven't had a chance to search for more instances of the bug! It happened in each bible I created with your program.

See images...

Attached Files

•  Screenshot_2015-01-22-17-03-56.png   665.19K   58 downloads
•  Screenshot_2015-01-22-17-04-39.png   596.3K   35 downloads

[Aaron Viscichini]

Hi GreenRaccoon,
First of all, awesome job! The stand alone is amazing.
However, there is a strange bug I found when doing a bible study on 2 Timothy 3 using bibles created by your script. In 2 Timothy 3 and 4 there are verses copied from the end of the chapter which are inserted into the beginning of the chapter. I'm not sure why and I haven't had a chance to search for more instances of the bug! It happened in each bible I created with your program.
See images...

I figured out that the bug appears in every chapter of every book that is part of a series of books (e.g. 1 Samuel, 2 Samuel, 1 Timothy, 2 Timothy, 1 Thessalonians, 2 Thessalonians, etc.). Apparently the additional number in the title of the book is the culprit and is confusing the programming algorithm. That's my guess.

Thanks for your hard work.

[GreenRaccoon23]

I figured out that the bug appears in every chapter of every book that is part of a series of books (e.g. 1 Samuel, 2 Samuel, 1 Timothy, 2 Timothy, 1 Thessalonians, 2 Thessalonians, etc.). Apparently the additional number in the title of the book is the culprit and is confusing the programming algorithm. That's my guess.

Thanks for your hard work.

Ok thanks I'll look into it tonight. I'm sure it's just a little, dumb mistake I made.

Edited by GreenRaccoon23, 25 January 2015 - 05:58 PM.

[GreenRaccoon23]

Hi GR,
 
I've tried the new code. Still unable to download. I'm on a Mac OS X Yosemite.
 
For the Expanded Bible (EXB), I get the following error:
Traceback (most recent call last):
File "/Users/***/Downloads/bible_gateway1.py", line 569, in <module>
translation_name = Copyright(translation).search(base_url)
File "/Users/***/Downloads/bible_gateway1.py", line 156, in search
self.publisher_info = soup.p.a.text
AttributeError: 'NoneType' object has no attribute 'a'
And for other bible, such as KJV, ESV
Traceback (most recent call last):
File "/Users/***/Downloads/bible_gateway1.py", line 569, in <module>
translation_name = Copyright(translation).search(base_url)
File "/Users/***/Downloads/bible_gateway1.py", line 158, in search
self.accept_me()
File "/Users/***/Downloads/bible_gateway1.py", line 179, in accept_me
''.join((self.copyright_info_bak.strip())),
TypeError: sequence item 0: expected str instance, int found

 
Whoa, I'm not sure how I didn't catch that. Maybe I uploaded the wrong file or something. I'll fix that tonight.
 
I would compile my Go package ('bgmysword') for Mac for you, but there's a bug in the current version of Go with building programs for a different operating system. Once the next version of Go comes out, I'll be able to upload a Mac package. 
 
 
EDIT
Fixed!

Also, I just had an idea. You could try this:   bgmysword.zip   2.86MB   14 downloads It's for Linux but it MIGHT work for Mac. 

Edited by GreenRaccoon23, 25 January 2015 - 06:37 PM.

[Tyler Gorrell]

Great app. Thanks for posting it.

[dyan]

 
Whoa, I'm not sure how I didn't catch that. Maybe I uploaded the wrong file or something. I'll fix that tonight.
 
I would compile my Go package ('bgmysword') for Mac for you, but there's a bug in the current version of Go with building programs for a different operating system. Once the next version of Go comes out, I'll be able to upload a Mac package. 
 
 
EDIT
Fixed!

Also, I just had an idea. You could try this: bgmysword.zip It's for Linux but it MIGHT work for Mac. 

 

Hi GR,

Tried the bgmysword.zip, could not get it to work for Mac. Looking forward to the Mac version once the next version of Go is released. 

 

I tried your script (bible_gateway.py.zip for linux & mac) with partial success. I was trying to download the EXB (Expanded Bible). Got this error again.

Finding EXB copyright information...
Traceback (most recent call last):
  File "/Users/****/Downloads/bible_gateway.py", line 558, in <module>
    translation_name = Copyright(translation).search(base_url)
  File "/Users/****/Downloads/bible_gateway.py", line 153, in search
    self.copyright_info = soup.p.text
AttributeError: 'NoneType' object has no attribute 'text'

I think this is what the problem was, but I could be wrong as I've no knowledge of coding etc.

Your script has this on line 147

                                          {'class': 'publisher-info-bottom'})

But the page for EXB on biblegateway has this

                                          {'class': 'publisher-info-bottom with-bga'})

 

So I changed that in your script, and it worked great, it started the download....until around book 22, when I got the following errors.

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 518, in urlopen
    body=body, headers=headers)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 322, in _make_request
    self._validate_conn(conn)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 727, in _validate_conn
    conn.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 204, in connect
    conn = self._new_conn()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 134, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 64, in create_connection
    for res in socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM):
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 530, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/adapters.py", line 370, in send
    timeout=timeout
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 564, in urlopen
    _pool=self, _stacktrace=stacktrace)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/util/retry.py", line 245, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/packages/six.py", line 309, in reraise
    raise value.with_traceback(tb)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 518, in urlopen
    body=body, headers=headers)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 322, in _make_request
    self._validate_conn(conn)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 727, in _validate_conn
    conn.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 204, in connect
    conn = self._new_conn()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 134, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 64, in create_connection
    for res in socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM):
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 530, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
requests.packages.urllib3.exceptions.ProtocolError: ('Connection aborted.', gaierror(8, 'nodename nor servname provided, or not known'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/****/Downloads/bible_gateway.py", line 600, in <module>
    SwordMe(num_book_chap, base_url)
  File "/Users/****/Downloads/bible_gateway.py", line 236, in __init__
    self.iter_me_chap(num, book, chap, base_url)
  File "/Users/****/Downloads/bible_gateway.py", line 252, in iter_me_chap
    sql_tuple = self.parse_me(book_num, curr_chap_num, url)
  File "/Users/****/Downloads/bible_gateway.py", line 269, in parse_me
    soup, soup_footnotes = self.fetch_me(url)
  File "/Users/****/Downloads/bible_gateway.py", line 259, in fetch_me
    content = requests.get(url, verify=True, timeout=20)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/api.py", line 65, in get
    return request('get', url, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/api.py", line 49, in request
    response = session.request(method=method, url=url, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/sessions.py", line 461, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/adapters.py", line 415, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', gaierror(8, 'nodename nor servname provided, or not known'))

I have no idea what they mean, and I'm sorry to keep troubling you, maybe it's Mac related. If this can be easily solved, that would be great, but if it's too much trouble, that's fine too, I'll just wait for your bgmysword for mac.

 

Thanks

[GreenRaccoon23]

I figured out that the bug appears in every chapter of every book that is part of a series of books (e.g. 1 Samuel, 2 Samuel, 1 Timothy, 2 Timothy, 1 Thessalonians, 2 Thessalonians, etc.). Apparently the additional number in the title of the book is the culprit and is confusing the programming algorithm. That's my guess.

Thanks for your hard work.

Ok, sorry about the delay. It was a pretty easy fix but it helped me find a bunch of other bugs. I've finished ironing everything out and I'll upload the new version.

[GreenRaccoon23]

 

Hi GR,

Tried the bgmysword.zip, could not get it to work for Mac. Looking forward to the Mac version once the next version of Go is released. 

 

I tried your script (bible_gateway.py.zip for linux & mac) with partial success. I was trying to download the EXB (Expanded Bible). Got this error again.

...

I think this is what the problem was, but I could be wrong as I've no knowledge of coding etc.

Your script has this on line 147

                                          {'class': 'publisher-info-bottom'})

But the page for EXB on biblegateway has this

                                          {'class': 'publisher-info-bottom with-bga'})

 

So I changed that in your script, and it worked great, it started the download....until around book 22, when I got the following errors.

...

I have no idea what they mean, and I'm sorry to keep troubling you, maybe it's Mac related. If this can be easily solved, that would be great, but if it's too much trouble, that's fine too, I'll just wait for your bgmysword for mac.

 

Thanks

 

Hey nice job figuring that out!

 

If I were you, I'd try running the 'bible_gateway.py' script again. It just means that it couldn't load the webpage. That happens to me sometimes too. (When it's loading a million webpages, it's not unlikely that one of them will mess up.) I added a workaround for this in the 'bgmysword' program, but unfortunately there's not a way to do it for the 'bible_gateway.py' script.

 

I'll keep looking to see if I can get it set up for Mac. It might be pretty easy to make instructions to install it from my github page.

 

 EDIT

It looks like it'd be pretty easy for you to make a Mac version of the 'bgmysword' program yourself. This would involve you downloading the code for it, installing a program called 'gcc', installing Google's 'Go' program, and then running a couple commands in Terminal. It sounds like a lot, but it should be easy. (For Windows it's a nightmare though. ) I'll show you how to do it.

1. Follow steps 1 and 2 from here to install 'gcc': http://www.cyberciti...l-gcc-compiler/
2. Download 'Go 1.4.1' from here: https://golang.org/dl/. The Mac versions start with 'darwin'. 'amd64' means 64-bit and '386' means 32-bit.
3. Install it. In case you get stuck, this page might help: http://www.cyberciti...ge-on-mac-os-x/
4. Then run these two commands in Terminal:
   

   go get github.com/GreenRaccoon23/bgmysword
   go install github.com/GreenRaccoon23/bgmysword
   

5. Then run it in Terminal like this:
   

   bgmysword

Edited by GreenRaccoon23, 03 February 2015 - 04:38 PM.

[dyan]

Hey GR,

 

Thanks for the excellent instructions. I managed to use both the script as well as bgmysword on the Mac. Both worked very well.

 

Finally got the EXB into mysword format. The only problem is that there are a total of 47660 verses, instead of the standard 31102 verses.

The script is somehow breaking up the verses at the wrong point. I've opened the file in SQLite Manager and compared it with a KJV bible and found this to be true.

 

Please help!

 

Thanks