Re: twisted.web with dynamic + static content

Jeff Grimmett <[email protected]> Mon, 19 Nov 2018 18:06:49 -0500
Newsgroups	gmane.comp.python.twisted
Message-ID	<CA+1iZEfN9AN-C4WyrrpkernVUo-9w=MxBMTmPN0X5efZP2QRgw@mail.gmail.com>
Glyph,

Belated (I was in the thick of a few unrelated crises) thanks - it helps to
understand a little bit of the background on the way it works.  I have made
adjustments and everything works the way I was hoping for now.

I wish I understood it a little better, if for no other reason than to
contribute some documentation tweaks.  Just how would that work, anyway?
The contribution model for Twisted is a little opaque from where I am
sitting. I'd love to help in areas where I am actually capable of doing :)

Regards,

Jeff


On Thu, Nov 1, 2018 at 3:44 AM Glyph <[email protected]> wrote:

> Hi Jeff,
>
> Thanks for using Twisted.
>
> Here's a version with some small changes that works, and is self-contained.
>
> import sys
>
> from twisted.internet import reactor, endpoints
> from twisted.web import server
> from twisted.web.resource import Resource
> from twisted.web.static import Data
>
> sys.path.append('lib')
>
> content = """
> <!DOCTYPE html>
> <html lang="en">
> <head>
>     <meta charset="UTF-8">
>     <link rel="stylesheet" href="/static/test.css" type="text/css" />
> </head>
> <body>
>     <span class='twistedTest'>This</span> is a test
> </body>
> </html>
> """
>
> class tServer(Resource):
>     def render_GET(self, request):
>         return bytes(content, "utf-8")
>
> if __name__ == "__main__":
>     root = Resource()
>     static_collection = Resource()
>     static_collection.putChild(b"test.css", Data(b".twistedTest {color:
> red;}", "text/css"))
>     root.putChild(b"static", static_collection)
>     root.putChild(b"", tServer())
>
>     site = server.Site(root)
>     endpoint = endpoints.TCP4ServerEndpoint(reactor, 8080)
>     endpoint.listen(site)
>
>     reactor.run()
>     print("Shutting down!")
>
>
> The problem with your first version was 'isLeaf', as Donal suggested.
> However, the problem was not simply that the flag was set, but rather what
> the flag means, and why it works that way.
>
> The root resource in any web server is a *collection*.  Which is to say,
> under normal circumstances, the root resource never has render_* invoked on
> it; you can't render it, because it's impossible, in the HTTP protocol, to
> spell a URL that doesn't start with "/".
>
> isLeaf changes this, and says "this resource is responsible for rendering
> all of its children; traversal stops here".  That means that it starts
> invoking render_GET to render "/", but also to render *every other path
> on the server*, including (unfortunately for you) /static/test.css.
>
> The modified example above instead uses a Resource() as the collection,
> and inserts a '' child for the index, and a separate 'static' child for the
> static index.  You can use a static.File for a directory here instead of a
> static resource, and anywhere you see putChild, you could also use a
> dynamic resource which overrides getChild to return the object rather than
> inserting it in advance.
>
> Of course, you might wonder what the point of 'isLeaf' is if it short
> circuits this stuff and makes it impossible to tell the difference between
> resources.
>
> Given that you have a directory, you want to use a static.File child
> resource and almost certainly *don't* want to set isLeaf; however, you
> might be wondering how one would even use isLeaf if it just cuts off the
> ability to tell the difference between resources.  The documentation on
> this is not great - it doesn't even appear as an attribute in the API
> reference, just an oblique reference in the docstring for
> https://twistedmatrix.com/documents/current/api/twisted.web.resource.Resource.html#getChild.
> But, the 'prepath' and 'postpath' attributes, lists of bytes, will tell you
> about where in the request traversal cycle you are, and allow you to
> distinguish which content to render directly within the body of render_*,
> rather than having to route to the right object using Twisted APIs.  So
> here's a working version with isLeaf=True:
>
> import sys
>
> from twisted.internet import reactor, endpoints
> from twisted.web import server
> from twisted.web.resource import Resource
>
> content = """
> <!DOCTYPE html>
> <html lang="en">
> <head>
>     <meta charset="UTF-8">
>     <link rel="stylesheet" href="/static/test.css" type="text/css" />
> </head>
> <body>
>     <span class='twistedTest'>This</span> is a test
> </body>
> </html>
> """
>
> css = """
> .twistedTest {
>     color: red;
> }
> """
>
> class tServer(Resource):
>     isLeaf = True
>     def render_GET(self, request):
>         if request.postpath == [b'']:
>             request.setHeader("content-type", "text/html")
>             return bytes(content, "utf-8")
>         elif request.postpath == [b'static', b'test.css']:
>             request.setHeader("content-type", "text/css")
>             return bytes(css, 'utf-8')
>         else:
>             request.setResponseCode(404)
>             return b'not found'
>
> if __name__ == "__main__":
>     site = server.Site(tServer())
>     endpoint = endpoints.TCP4ServerEndpoint(reactor, 8080)
>     endpoint.listen(site)
>
>     reactor.run()
>     print("Shutting down!")
>
>
> I hope this clears up the request traversal model a little bit.
>
> -glyph
>
>
> On Oct 31, 2018, at 2:15 PM, Jeff Grimmett <[email protected]> wrote:
>
> Tried that, I get a big
>
> No Such Resource
>
> No such child resource.
> back.  Watching it in FF's development panel, I see a 404 come back for
> /.  /static doesn't get served at all, of course.
>
> This, however, DID work.
>
> class tServer(Resource):
>     isLeaf = False
>
>     def getChild(self, path, request):
>         print('You know what you doing.')
>
>         if path == b'':
>             print("Rendering /")
>             return self
>
>         return Resource.getChild(self, path, request)
>
>     def render_GET(self, request):
>         return bytes(content, "utf-8")
>
> (ignore my printf debugging plz)
>
> So, Thanks! :)
>
> Regards,
>
> Jeff
>
>
> On Tue, Oct 30, 2018 at 6:42 PM Donal McMullan <[email protected]>
> wrote:
>
>> Try replacing:
>> isLeaf = True
>> with
>> isLeaf = False
>>
>>
>> On Tue, 30 Oct 2018 at 21:32, Jeff Grimmett <[email protected]> wrote:
>>
>>> I'm sure I'm overlooking something obvious here but I just can't get my
>>> head around it.
>>>
>>> Here's the setup: twisted.web server that generates dynamic content.
>>> Child that serves up static content, e.g. css and favoicon.  However, the
>>> static content isn't making it. Instead, any hit to localhost/static
>>> actually yields up a copy of / again.
>>>
>>> Here's the server code
>>>
>>> import sys
>>>
>>> from twisted.internet import reactor, endpoints
>>> from twisted.web import server
>>> from twisted.web.resource import Resource
>>> from twisted.web.static import File
>>>
>>> sys.path.append('lib')
>>>
>>> content = """
>>> <!DOCTYPE html>
>>> <html lang="en">
>>> <head>
>>>     <meta charset="UTF-8">
>>>     <link rel="stylesheet" href="/static/test.css" type="text/css" />
>>> </head>
>>> <body>
>>>     <span class='twistedTest'>This</span> is a test
>>> </body>
>>> </html>
>>> """
>>>
>>>
>>> class tServer(Resource):
>>>     isLeaf = True
>>>
>>>     def render_GET(self, request):
>>>         return bytes(content, "utf-8")
>>>
>>>
>>> if __name__ == "__main__":
>>>     root = tServer()
>>>     root.putChild(b"static", File("static"))
>>>
>>>     site = server.Site(root)
>>>     endpoint = endpoints.TCP4ServerEndpoint(reactor, 8080)
>>>     endpoint.listen(site)
>>>
>>>     reactor.run()
>>>     print("Shutting down!")
>>>
>>> It's run with the command 'python tserver.py'.  The expectation is that
>>> what is inside the custom <span> will be red.
>>>
>>> In the same dir as the script is a subdir 'static' with the css file
>>> inside it.
>>>
>>> If I replace 'root' with     root = Resource() then / doesn't serve up
>>> anything, but /static is a directory listing of the static directory.
>>>
>>> The dynamic server is basically a copy of several tutorials cooked down
>>> to something that I could use to demonstrate the problem.
>>>
>>> What am I missing here? /headscratch
>>>
>>> Regards,
>>>
>>> Jeff
>>> _______________________________________________
>>> Twisted-Python mailing list
>>> [email protected]
>>> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>>
>> _______________________________________________
>> Twisted-Python mailing list
>> [email protected]
>> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
> _______________________________________________
> Twisted-Python mailing list
> [email protected]
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
> _______________________________________________
> Twisted-Python mailing list
> [email protected]
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>

_______________________________________________
Twisted-Python mailing list
[email protected]
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python