Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

macOS Help with Apple Script for Yahoo weather and Geektool

G

gr4z

macrumors 6502
Original poster
Aug 7, 2010
318
48
England
Hi

Now that the Yahoo APIs have changed myt original script fails to work.

I have the following XML that I would like to use with Geektool, but am having issues :

<yweather:location xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" city="Peterborough" country="United Kingdom" region=" England"/>
<yweather:wind xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" chill="64" direction="220" speed="11"/>
<yweather:atmosphere xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" humidity="91" pressure="1009.0" rising="0" visibility="13.9"/>
<yweather:astronomy xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" sunrise="4:37 am" sunset="9:28 pm"/>
<yweather:condition xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="11" date="Mon, 20 Jun 2016 01:00 PM BST" temp="64" text="Showers"/>
<yweather:forecast xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="12" date="20 Jun 2016" day="Mon" high="65" low="58" text="Rain"/>

I dont really know Apple Script very well but my old script uses the following code to extract the data for the forecast:

set forecast1Conditions to (do shell script "awk '/yweather:forecast/{print $0}' " & weatherFile & "| awk 'NR==1{print;exit}' | sed -e 's/.text=//;s/code=.//' | sed 's/\"//g'") as string
set forecast1Conditions to (my trim_string(forecast1Conditions, white_space, "both"))
do shell script ("echo " & forecast1Conditions)

This bombs out now with the newly formatted XML. Can anyone explain what am I doing wrong in trying to select the text="Rain" from the final yweather:forecast ?? I have plenty of other lines selecting all sorts of stuff so if someone can help with this I should be able to work out what I am doing wrong. I dont really understand AWK and SED.

Thanks

Apologies posted this in the UI customisations forum first...
 
You can start by breaking that whole do shell script line into pieces and see what each piece produces as output. There are a lot of online tutorials to be found about awk and sed. Perhaps it's time to check those out or look into an XML parser.
 
OK I am now using the following XML code from a file called weatherfile

<yweather:condition xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="30" date="Mon, 20 Jun 2016 08:00 PM BST" temp="18" text="Partly Cloudy"/>
<yweather:forecast xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="11" date="20 Jun 2016" day="Mon" high="18" low="13" text="Showers"/>
<yweather:forecast xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="28" date="21 Jun 2016" day="Tue" high="20" low="12" text="Mostly Cloudy"/>
<yweather:forecast xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="28" date="22 Jun 2016" day="Wed" high="20" low="12" text="Mostly Cloudy"/>
<yweather:forecast xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="12" date="23 Jun 2016" day="Thu" high="20" low="14" text="Rain"/>
<yweather:forecast xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="30" date="24 Jun 2016" day="Fri" high="19" low="13" text="Partly Cloudy"/>
<yweather:forecast xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="30" date="25 Jun 2016" day="Sat" high="17" low="12" text="Partly Cloudy"/>
<yweather:forecast xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="30" date="26 Jun 2016" day="Sun" high="18" low="12" text="Partly Cloudy"/>
<yweather:forecast xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="30" date="27 Jun 2016" day="Mon" high="18" low="13" text="Partly Cloudy"/>
<yweather:forecast xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="28" date="28 Jun 2016" day="Tue" high="18" low="12" text="Mostly Cloudy"/>
<yweather:forecast xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="28" date="29 Jun 2016" day="Wed" high="18" low="12" text="Mostly Cloudy"/>

However, trying to extract data from the first line using

do shell script "grep -E 'yweather:condition' " & weatherFile & "| sed -e 's/.*text=//;s/\\/>.*//' | sed 's/\"//g'"

Results in the text from the last line, 'mostly cloudy'? It should be from the line yweather:condition and the text should be 'Partly Cloudy'.

I don't understand why. Can anyone help?
 
Try using:

do shell script "grep -E 'yweather:condition' " & weatherFile & "| sed -e 's/.*text=//;s/\/>.*//' | sed 's/\"//g'"

You have too many backslash characters in the second sed edit command.
 
Using

do shell script "grep -E 'yweather:condition' " & weatherFile & "| sed -e 's/.*text=//;s/\/>.*//' | sed 's/\"//g'"

And the script moans saying expected """ but found unknown token where the \ was removed. Am confused now.
[doublepost=1466533347][/doublepost]OK looking into this further I have found the input XML weatherfile, does not have any carriage returns at the end of each line. So it looks like

<yweather:condition xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="30" date="Mon, 20 Jun 2016 08:00 PM BST" temp="18" text="Partly Cloudy"/><yweather:forecast xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="11" date="20 Jun 2016" day="Mon" high="18" low="13" text="Showers"/><yweather:forecast xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" code="28" date="21 Jun 2016" day="Tue" high="20" low="12" text="Mostly Cloudy"/>

If I manually edit the CRs in my scripts works. A quick google on how to insert a carriage return after /> and before <yweather throws up some really confusing SED scripts. Anyone have a one liner than can do this?
 
Mmm I want to change

/><yweather

To

/>
<yweather

Not sure this link helps much to be honest as its not a straight forward character swap. I need to insert a carriage return.
 
Parsing XML with regular expressions is horrible. To be fair though, XML is horrible in of itself.

Generally speaking you are best off using XPath for these kind of things. Needless to say it is a joy but better than REs for anything but the most simple extraction.

There are lots of command line xpath tools available. For the purpose of this example I will use /usr/bin/xpath because I think it comes preinstalled with OSX - at least I seem to have it. If you don't have it you can get it via e.g. macports when you install Perl. Alternatively see if you have xmllint with has an xpath query feature.

The query
Code: 
/yweather:condition/@text

Selects the text attribute of each yweather:condition element. Wrap that in the string function to get just the value (and not the attribute name) and we have:
Code: 
string(/yweather:condition/@text)

Now of course there are some complications. The data given above is not really one XML document (which would have an opening and closing tag surrounding everything) but more like standalone documents (all consisting of just one tag) on each line. Maybe you have extracted this data from something larger? If so then stop and you can skip this step. If not then we have to make sure we feed each XML document to xpath one at a time. If you only ever need the first line then just use "head -1" but assuming you don't know which line you want then we can use a loop:

Code: 
while read -r xml ; do echo "${xml}" | xpath "string(/yweather:condition/@text)" 2>/dev/null ; done < weatherFile

The redirection to /dev/null hides some meta output form the tool. It probably wouldn't be necessary if you used a different tool. One other downside of this tool is it will print out new lines even if it found nothing. If you want to suppress those then a quick fix is to use tr. This leaves you with:

Code: 
while read -r xml ; do echo "${xml}" | xpath "string(/yweather:condition/@text)" 2>/dev/null | tr -d '\n' ; done < weatherFile

Then you just need to call that from your AppleScript.
 
  • Like
Reactions: kryten2
It's not entirely clear to me exactly what's in this file you're trying to process, but if it is indeed all on one line with no hidden characters between the /> and < strings, you can use something like the following to insert newline characters where wanted:

Code: 
sed -e 's/></>[</g FILENAME | tr '[' '\n' | REST OF CODE HERE

The sed command inserts a dummy character between the characters that end one line and start the next, and the tr command (which can only deal with single character changes) changes that to a newline character. The dummy character needs to be some character that you don't expect to ever see in the file you're parsing. This is a kludge, but it should work if the file contents are as you've stated and if you can find a workable dummy character. You should be able to do this with just a sed command, but that doesn't appear to work as expected in the OS X sed implementation. A master at designing the sort of regular expressions (regex) used by sed could probably come up with some better approach (or maybe using awk), but more information would be needed about the expected contents of the files you're processing to come up with a bulletproof approach.

If you are truly dealing with a correctly formated XML file, which is not at all clear to me you are, using an XML parser is a cleaner way to do this, although it will take some spin-up on your part if you want to understand what it's doing.
 
Don't know what kind of file the OP is using but after poking around a bit at yahoo this is what I got.

Code: 
<?xml version="1.0" encoding="UTF-8"?>
<query xmlns:yahoo="http://www.yahooapis.com/v1/base.rng"
    yahoo:count="1" yahoo:created="2016-06-20T21:25:55Z" yahoo:lang="en-US">
    <results>
        <channel>
            <yweather:units
                xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                distance="mi" pressure="in" speed="mph" temperature="F"/>
            <title>Yahoo! Weather - St Nicolas, Oost-Vlaanderen, BE</title>
            <link>http://us.rd.yahoo.com/dailynews/rss/weather/Country__Country/*https://weather.yahoo.com/country/state/city-977271/</link>
            <description>Yahoo! Weather for St Nicolas, Oost-Vlaanderen, BE</description>
            <language>en-us</language>
            <lastBuildDate>Mon, 20 Jun 2016 11:25 PM CEST</lastBuildDate>
            <ttl>60</ttl>
            <yweather:location
                xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                city="St Nicolas" country="Belgium" region=" Oost-Vlaanderen"/>
            <yweather:wind
                xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                chill="59" direction="230" speed="14"/>
            <yweather:atmosphere
                xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                humidity="90" pressure="1014.0" rising="0" visibility="16.1"/>
            <yweather:astronomy
                xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                sunrise="5:28 am" sunset="10:3 pm"/>
            <image>
                <title>Yahoo! Weather</title>
                <width>142</width>
                <height>18</height>
                <link>http://weather.yahoo.com</link>
                <url>http://l.yimg.com/a/i/brand/purplelogo//uh/us/news-wea.gif</url>
            </image>
            <item>
                <title>Conditions for St Nicolas, Oost-Vlaanderen, BE at 10:00 PM CEST</title>
                <geo:lat xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#">51.164459</geo:lat>
                <geo:long xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#">4.1415</geo:long>
                <link>http://us.rd.yahoo.com/dailynews/rss/weather/Country__Country/*https://weather.yahoo.com/country/state/city-977271/</link>
                <pubDate>Mon, 20 Jun 2016 10:00 PM CEST</pubDate>
                <yweather:condition
                    xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                    code="26" date="Mon, 20 Jun 2016 10:00 PM CEST"
                    temp="60" text="Cloudy"/>
                <yweather:forecast
                    xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                    code="11" date="20 Jun 2016" day="Mon" high="60"
                    low="57" text="Showers"/>
                <yweather:forecast
                    xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                    code="28" date="21 Jun 2016" day="Tue" high="67"
                    low="59" text="Mostly Cloudy"/>
                <yweather:forecast
                    xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                    code="30" date="22 Jun 2016" day="Wed" high="75"
                    low="61" text="Partly Cloudy"/>
                <yweather:forecast
                    xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                    code="4" date="23 Jun 2016" day="Thu" high="78"
                    low="65" text="Thunderstorms"/>
                <yweather:forecast
                    xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                    code="4" date="24 Jun 2016" day="Fri" high="72"
                    low="65" text="Thunderstorms"/>
                <yweather:forecast
                    xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                    code="30" date="25 Jun 2016" day="Sat" high="70"
                    low="59" text="Partly Cloudy"/>
                <yweather:forecast
                    xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                    code="30" date="26 Jun 2016" day="Sun" high="67"
                    low="58" text="Partly Cloudy"/>
                <yweather:forecast
                    xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                    code="30" date="27 Jun 2016" day="Mon" high="67"
                    low="58" text="Partly Cloudy"/>
                <yweather:forecast
                    xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                    code="30" date="28 Jun 2016" day="Tue" high="66"
                    low="58" text="Partly Cloudy"/>
                <yweather:forecast
                    xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0"
                    code="30" date="29 Jun 2016" day="Wed" high="69"
                    low="58" text="Partly Cloudy"/>
                <description>&lt;![CDATA[&lt;img src="http://l.yimg.com/a/i/us/we/52/26.gif"/&gt;
&lt;BR /&gt;
&lt;b&gt;Current Conditions:&lt;/b&gt;
&lt;BR /&gt;Cloudy
&lt;BR /&gt;
&lt;BR /&gt;
&lt;b&gt;Forecast:&lt;/b&gt;
&lt;BR /&gt; Mon - Showers. High: 60Low: 57
&lt;BR /&gt; Tue - Mostly Cloudy. High: 67Low: 59
&lt;BR /&gt; Wed - Partly Cloudy. High: 75Low: 61
&lt;BR /&gt; Thu - Thunderstorms. High: 78Low: 65
&lt;BR /&gt; Fri - Thunderstorms. High: 72Low: 65
&lt;BR /&gt;
&lt;BR /&gt;
&lt;a href="http://us.rd.yahoo.com/dailynews/rss/weather/Country__Country/*https://weather.yahoo.com/country/state/city-977271/"&gt;Full Forecast at Yahoo! Weather&lt;/a&gt;
&lt;BR /&gt;
&lt;BR /&gt;
(provided by &lt;a href="http://www.weather.com" &gt;The Weather Channel&lt;/a&gt;)
&lt;BR /&gt;
]]&gt;</description>
                <guid isPermaLink="false"/>
            </item>
        </channel>
    </results>
</query>
 
kryten2 said:
Don't know what kind of file the OP is using but after poking around a bit at yahoo this is what I got.
Click to expand...

So, looks like bog-standard XML with line-feeds and everything. To OP: have you been posting files as you've received them (such as what kryten2 just posted) or after the file has been passed through some earlier processing? If your initial file is well-formatted XML, then a formal XML parser is a much safer way to go than a roll-your-own parser.
 
Thanks for all your great help here guys. The XML is stored using the following command

set weatherCurl to (do shell script "curl --silent 'https://query.yahooapis.com/v1/publ...ecast WHERE woeid= 22961 and u="c"&format=xml' | grep 'yweather:' >" & weatherFile2)

Obviously weatherCurl and weatherFile2 are defined elsewhere. weatherFile2 is a XML file but there is not any carriage returns anywhere. Not sure why.

I am not much of a coder I must admit and have used the following crap code to force a carriage return

do shell script "tr '<' '
' < " & weatherFile2 & "> " & weatherFile

This seems to work my original Yahoo script and now I get the attached on my desktop. So its working albeit with my crude code.
 

Attachments

  • weathericon.png
    weathericon.png
    111 KB · Views: 244
Wasn't sure that what you did (just brute-force enter a carriage return) would work, and I wasn't sure you wanted to sacrifice the "<" character the way you have done. Give the following a try:

Code: 
set forecast1Conditions to ( do shell script "curl --silent " & wxURL & " tr '<' '\n' | grep condition | sed -e 's/.*text="//;s/"\/>//' )

where wxURL is the URL for the file that you want to process. The 'grep' section picks out the single line containing 'condition', the first 'sed' edit gets rid of all characters up to and including the " at the start of the conditions text string, and the second edit gets rid of the three characters at the end of the line. Using the \n newline character makes what you're doing a bit more transparent when you look at this a year from now. I've never been happy putting a "naked" return in a script, and that's what the \n metacharacter is designed to avoid. (I'm not an Applescript user so I've tested this only within Terminal. There may be some tweaking needed to get things like ' and " sorted out correctly, particularly in the sed commands section.)
 
Code: 
curl --silent 'https://query.yahooapis.com/v1/public/yql?q=SELECT%20*%20FROM%20weather.forecast%20WHERE%20woeid%3D%2022961%20and%20u%3D%22c%22&format=xml' | xpath "string(/query/results/channel/item/yweather:condition/@text)" 2>/dev/null
 
cqexbesd said:
Code: 
curl --silent 'https://query.yahooapis.com/v1/public/yql?q=SELECT%20*%20FROM%20weather.forecast%20WHERE%20woeid%3D%2022961%20and%20u%3D%22c%22&format=xml' | xpath "string(/query/results/channel/item/yweather:condition/@text)" 2>/dev/null
Click to expand...

Thanks for this, what would the code be if I wanted @text, @temp?
 
gr4z said:
Thanks for this, what would the code be if I wanted @text, @temp?
Click to expand...

If you mean you want both values then:

Code: 
"/query/results/channel/item/yweather:condition/@*[name()='text' or name()='temp']"

should work - but you would need to parse the output - it won't be a comma separated list. To get XPath to do the concatenation you would need to use XPath 2 which doesn't seem to be supported by /usr/bin/xpath (at least under 10.9).

If you only want those 2 values though, you know you can just ask for that from Yahoo by changing the query to be:

Code: 
https://query.yahooapis.com/v1/public/yql?q=select%20item.condition.temp%2C%20item.condition.text%20%20from%20weather.forecast%20where%20woeid%20%3D%2022961%20and%20u%3D%22c%22&format=xml&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys

and with simpler XML it becomes feasible to use AppleScript built in XML handling. e.g.

Code: 
set http_response to do shell script "curl -s https://query.yahooapis.com/v1/public/yql?q=select%20item.condition.temp%2C%20item.condition.text%20%20from%20weather.forecast%20where%20woeid%20%3D%2022961%20and%20u%3D%22c%22&format=xml&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys"

tell application "System Events"
    set xml_root to make new XML data with data http_response
    set conditions to the XML element "yweather:condition" of XML element "item" of XML element "channel" of XML element "results" of XML element "query" of xml_root
  
    set temp to the value of the XML attribute "temp" of conditions
    set desc to the value of the XML attribute "text" of conditions
end tell

error handling left as an exercise for the reader. NB text is called desc because text is reserved.

I would say though, the more I see AppleScript the more I recommend learning a UNIX based scripting language where these things are easier. Personally I use a lot of Perl but there are many choices, most of which aren't wrong.
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back