Jump to content

Please read the Forum Rules before posting.

Photo

BibleGateway to MySword


  • Please log in to reply
50 replies to this topic

#11 GreenRaccoon23

GreenRaccoon23

    Liking e-Sword

  • Members
  • PipPip
  • 19 posts
  • LocationIowa/Nebraska, United States
Offline

Posted 14 January 2015 - 09:51 AM

Noob here to C++/python ... have played with php some and Fortran 77 back in the day LOL ... I'm stuck installing lxml.

I've installed Visual Studio C++ Express 2010, Python 3.4.2, BeautifulSoup and pip.  I can't get lxml to install ... looks like I'm missing libxml2 and libxslt.  I found pre-compiled libraries (libxml2-2.7.8.win32 & libxslt-1.1.26.win32) from http://xmlsoft.org/sources/win32/ and unpacked them, then put the paths to the bin folder into my Path, but still no luck.  I saw some windows installers for lxml but only for python 3.2

Windows 7 32 bit
Python 3.4.2

Modules installed:
beautifulsoup 4.3.2
pip 1.5.6
requests 2.5.1
setuptools 2.1
virtualenv 12.0.5

Here is a link to my error log: damartin.net\files\pip.log

Any suggestions?

lxml actually has its own Windows installer. Have you tried that? It's here: https://pypi.python....7a2fbfeb628e5d8

 

You actually don't need to compile lxml or put it in your bin path. Python has a cool little function to make a folder part of its path semi-automatically. All you need to do is put a blank file called "__init__.py" in the folder.

 

So if the "biblegateway_mysword.py" script is in your Downloads folder, this is what you'd do:

  1. Create a blank file (like a blank text file) in your Downloads folder and call it "__init__.py".
  2. Download the "tar.gz" (source) file from here. (The download link as of today is this.)
  3. Extract/unzip the "tar.gz" file. (If Windows can't unzip "tar.gz" files, 7zip is awesome and will do it.)
  4. Move the new "lxml" folder to the folder where you created the blank "__init__.py" file.
  5. Now python will automatically search this folder whenever you call "import", meaning that it will work with my script.

Or, if you want to make your Downloads folder look less messy, you could move those files (the "biblegateway_mysword.py" script, the "lxml" folder, and the "__init__.py" file) anywhere you want. I put mine in a folder called "python" inside of my User folder.

 

I've almost finished translating this script into Go, so you could also wait for me to finish and upload the Go script. It's like 10 times faster than the python script cause Go is just phenomenal (almost as fast as C yet almost as simple as python). Plus, Go is easier to install.


Edited by GreenRaccoon23, 14 January 2015 - 09:58 AM.

Lutheran sin|nerNinja in the Lutheran Ninja Clan.  :ph34r: http://www.worldview...ran-ninja-clan/

 

"You know less than does a log on the ground."

"Your words are so foolishly and ignorantly composed that I cannot believe you understand them."

-Martin Luther (who else?)


#12 Don Martin

Don Martin

    New to Bible Support

  • Members
  • Pip
  • 5 posts
Offline

Posted 14 January 2015 - 11:41 AM

Thanks GR ...

 

I tried that, but it still doesn't seem to be finding lxml.

 

Traceback (most recent call last):
  File "bible_gateway.py", line 558, in <module>
    translation_name = Copyright(translation).search(base_url)
  File "bible_gateway.py", line 152, in search
    parse_only=only_publisher_tag)
  File "C:\Python34\lib\site-packages\bs4\__init__.py", line 152, in __init__
    % ",".join(features))
bs4.FeatureNotFound: Couldn't find a tree builder with the features you requeste
d: lxml. Do you need to install a parser library?

 

Googling this ... it looks to be an issue with BeautifulSoup? I have version 4.3.2 which appears to be current.  Should I downgrade?

 

Ok ... I went down to Python 3.2 and was able to install lxml.

 

Here are my installed packages:

beautifulsoup4 (4.3.2)
lxml (3.4.1)
pip (6.0.6)
requests (2.5.1)
setuptools (11.3.1)
virtualenv (12.0.5)

 

Now I get this error message:

Traceback (most recent call last):
  File "bible_gateway.py", line 558, in <module>
    translation_name = Copyright(translation).search(base_url)
  File "bible_gateway.py", line 156, in search
    self.accept_me()
  File "bible_gateway.py", line 208, in accept_me
    [print(section) for section in license_agreement]
  File "bible_gateway.py", line 208, in <listcomp>
    [print(section) for section in license_agreement]
  File "C:\Python32\lib\encodings\cp437.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_map)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\xa9' in position 13
2: character maps to <undefined>


Edited by Don Martin, 14 January 2015 - 02:22 PM.


#13 dyan

dyan

    e-Sword Addict

  • Veterans
  • PipPipPipPip
  • 89 posts
Offline

Posted 14 January 2015 - 12:08 PM

Hi,

 

I'm trying to download a bible to convert to mysword bible. I keep getting this error while running the script:

Traceback (most recent call last):
  File "/Users/****/Downloads/bible_gateway.py", line 558, in <module>
    translation_name = Copyright(translation).search(base_url)
  File "/Users/****/Downloads/bible_gateway.py", line 153, in search
    self.copyright_info = soup.p.text
AttributeError: 'NoneType' object has no attribute 'text'

What should I do?



#14 GreenRaccoon23

GreenRaccoon23

    Liking e-Sword

  • Members
  • PipPip
  • 19 posts
  • LocationIowa/Nebraska, United States
Offline

Posted 17 January 2015 - 02:52 AM

Hi,

 

I'm trying to download a bible to convert to mysword bible. I keep getting this error while running the script:

Traceback (most recent call last):
  File "/Users/****/Downloads/bible_gateway.py", line 558, in <module>
    translation_name = Copyright(translation).search(base_url)
  File "/Users/****/Downloads/bible_gateway.py", line 153, in search
    self.copyright_info = soup.p.text
AttributeError: 'NoneType' object has no attribute 'text'

What should I do?

Hm that's odd. That means that it couldn't find the spot on the webpage that shows the copyright information. Which translation were you trying to download? I can't seem to find one that gives me that error.


Lutheran sin|nerNinja in the Lutheran Ninja Clan.  :ph34r: http://www.worldview...ran-ninja-clan/

 

"You know less than does a log on the ground."

"Your words are so foolishly and ignorantly composed that I cannot believe you understand them."

-Martin Luther (who else?)


#15 GreenRaccoon23

GreenRaccoon23

    Liking e-Sword

  • Members
  • PipPip
  • 19 posts
  • LocationIowa/Nebraska, United States
Offline

Posted 17 January 2015 - 02:54 AM

Thanks GR ...

 

I tried that, but it still doesn't seem to be finding lxml.

 

Traceback (most recent call last):
  File "bible_gateway.py", line 558, in <module>
    translation_name = Copyright(translation).search(base_url)
  File "bible_gateway.py", line 152, in search
    parse_only=only_publisher_tag)
  File "C:\Python34\lib\site-packages\bs4\__init__.py", line 152, in __init__
    % ",".join(features))
bs4.FeatureNotFound: Couldn't find a tree builder with the features you requeste
d: lxml. Do you need to install a parser library?

 

Googling this ... it looks to be an issue with BeautifulSoup? I have version 4.3.2 which appears to be current.  Should I downgrade?

 

Ok ... I went down to Python 3.2 and was able to install lxml.

 

Here are my installed packages:

beautifulsoup4 (4.3.2)
lxml (3.4.1)
pip (6.0.6)
requests (2.5.1)
setuptools (11.3.1)
virtualenv (12.0.5)

 

Now I get this error message:

Traceback (most recent call last):
  File "bible_gateway.py", line 558, in <module>
    translation_name = Copyright(translation).search(base_url)
  File "bible_gateway.py", line 156, in search
    self.accept_me()
  File "bible_gateway.py", line 208, in accept_me
    [print(section) for section in license_agreement]
  File "bible_gateway.py", line 208, in <listcomp>
    [print(section) for section in license_agreement]
  File "C:\Python32\lib\encodings\cp437.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_map)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\xa9' in position 13
2: character maps to <undefined>

Oh ok, I remember reading about that somewhere. I think that means that Windows won't let the script print in color. You COULD fix it by following the instructions here to enable the ANSI.SYS file, but that's kind of a ridiculous solution in my opinion. I'll just remove the colors in my next upload.


Lutheran sin|nerNinja in the Lutheran Ninja Clan.  :ph34r: http://www.worldview...ran-ninja-clan/

 

"You know less than does a log on the ground."

"Your words are so foolishly and ignorantly composed that I cannot believe you understand them."

-Martin Luther (who else?)


#16 GreenRaccoon23

GreenRaccoon23

    Liking e-Sword

  • Members
  • PipPip
  • 19 posts
  • LocationIowa/Nebraska, United States
Offline

Posted 17 January 2015 - 03:03 AM

I've translated this program into a different coding language because I wanted to make it a little easier to install it. It's like 10 times faster now too. I'm still working out a couple kinks to make it a stand-alone program, so I can't upload it yet. But in case anyone knows how to set up Golang packages, I'll post the link to the "source code" on my github account.

 

https://github.com/G...oon23/bgmysword


Lutheran sin|nerNinja in the Lutheran Ninja Clan.  :ph34r: http://www.worldview...ran-ninja-clan/

 

"You know less than does a log on the ground."

"Your words are so foolishly and ignorantly composed that I cannot believe you understand them."

-Martin Luther (who else?)


#17 GreenRaccoon23

GreenRaccoon23

    Liking e-Sword

  • Members
  • PipPip
  • 19 posts
  • LocationIowa/Nebraska, United States
Offline

Posted 19 January 2015 - 06:38 PM

Thanks GR ...

 

I tried that, but it still doesn't seem to be finding lxml.

 

Traceback (most recent call last):
  File "bible_gateway.py", line 558, in <module>
    translation_name = Copyright(translation).search(base_url)
  File "bible_gateway.py", line 152, in search
    parse_only=only_publisher_tag)
  File "C:\Python34\lib\site-packages\bs4\__init__.py", line 152, in __init__
    % ",".join(features))
bs4.FeatureNotFound: Couldn't find a tree builder with the features you requeste
d: lxml. Do you need to install a parser library?

 

Googling this ... it looks to be an issue with BeautifulSoup? I have version 4.3.2 which appears to be current.  Should I downgrade?

 

Ok ... I went down to Python 3.2 and was able to install lxml.

I found out that you can't install lxml with Python 3.4.2 because with the latest version of lxml, someone either forgot to upload or just didn't upload a 64-bit version. In case you want to install Python 3.4.2, I wrote a script that will download and install a 64-bit version of lxml.


Lutheran sin|nerNinja in the Lutheran Ninja Clan.  :ph34r: http://www.worldview...ran-ninja-clan/

 

"You know less than does a log on the ground."

"Your words are so foolishly and ignorantly composed that I cannot believe you understand them."

-Martin Luther (who else?)


#18 Don Martin

Don Martin

    New to Bible Support

  • Members
  • Pip
  • 5 posts
Offline

Posted 20 January 2015 - 02:47 PM

Thanks GR ... I got the 64 bit program to run.

 

I tried adding the dependencies to my 32-bit install, but I still got the same message about windows not allowing to print in color.  I did add ANSI.SYS.  Probably the best thing at this point is to reinstall and start from scratch.  I'll keep you posted.

 

-d



#19 dyan

dyan

    e-Sword Addict

  • Veterans
  • PipPipPipPip
  • 89 posts
Offline

Posted 21 January 2015 - 04:53 AM

Hm that's odd. That means that it couldn't find the spot on the webpage that shows the copyright information. Which translation were you trying to download? I can't seem to find one that gives me that error.

 

The version I was trying to download was the expanded bible.



#20 Don Martin

Don Martin

    New to Bible Support

  • Members
  • Pip
  • 5 posts
Offline

Posted 21 January 2015 - 12:24 PM

I did a clean install on my 32-bit machine using your dependency install script and it works ... your right, 64-bit is much faster

 

Thanks much!


Edited by Don Martin, 21 January 2015 - 01:31 PM.





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users