Re: Toaster Error - Rocko

Hi David,

1. Attached is the new log with obj.path and it does appear that all
occurrences have only case difference. How do we solve this?
2. I am not sure about files with Unicode in their names but one thing is
that we have meta-mono layer included and we are building that within the
image. I see the mono files sometimes have very long, non standard names.
3. I have mysql running in non-strict mode because otherwise it was
complaining about very long path in the build. Could this be the root cause
of the issue(s)?

Thanks for your help.

Best Regards,
Shakeel

On Tue, Aug 28, 2018 at 10:03 AM Reyna, David <[email protected]>
wrote:

> Hi Shakeel,
>
>
>
> 1. Since all of the offending objects are Target_File , you could change
> the log message so show the object’s actual value by adding “.path”.
>
>
>
>                    logger.info("    FOUND:'%s'" % (obj.path))
>
>
>
> That will show us to look inside of the Target_File objects and observe
> the actual multiple entries.
>
>
>
> Since we are seeing a lot of upper/lower case similar files, I wonder if
> it is a case problem:
>
>   (0-2-/usr/share/terminfo/a-18)({'target': <Target: Target object>,
> 'path': '/usr/share/terminfo/a', 'inodetype': 2})
>
>   (0-2-/usr/share/terminfo/A-18)({'target': <Target: Target object>,
> 'path': '/usr/share/terminfo/A', 'inodetype': 2})
>
>
>
> The only cost of these errors should be perhaps missing entries in the
> target file tree.
>
>
>
> 2. The second error set looks a lot nastier.
>
>
>
> _mysql_exceptions.OperationalError: (1267, "Illegal mix of collations
> (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation
> '='")
>
>
>
> Where did the Swedish text come from?
>
>
>
> Do you perhaps have files with Unicode in their names?
>
>
>
> I am wondering if we should have Toaster normalize all file paths/names
> for TargetFile objects to UTF8 or something. That is certainly doable.
>
>
>
> - David
>
>
>
> *From:* Muhammad Shakeel Khan [mailto:[email protected]]
> *Sent:* Monday, August 27, 2018 6:17 AM
> *To:* Reyna, David
> *Cc:* [email protected]
> *Subject:* Re: [Toaster] Toaster Error - Rocko
>
>
>
> Hi David,
>
>
>
> Thanks for looking into this.
>
>
>
> OK, I will ignore 1) as you said that the messages are largely passive.
>
>
>
> For 2) I have added following slightly modified code to _cached_get
>
>
>
>     def _cached_get(self, clazz, **kwargs):
>
>         """ This is a memory-cached get. We assume that the objects will
> not change  in the database between gets.
>
>         """
>
>         assert issubclass(clazz, models.Model), "_cached_get needs to get
> the class as first argument"
>
>
>
>         key = ORMWrapper._build_key(**kwargs)
>
>         dictname = "objects_%s" % clazz.__name__
>
>
>
>         if not dictname in vars(self).keys():
>
>             vars(self)[dictname] = {}
>
>
>
>         if not key in vars(self)[dictname].keys():
>
>             try:
>
>                 vars(self)[dictname][key] = clazz.objects.get(**kwargs)
>
>             except (ObjectDoesNotExist, MultipleObjectsReturned) as e:
>
>                 logger.info("ERROR:buildinfohelper:_cached_get: '%s'
> (%s)(%s)" % (e, key, kwargs))
>
>                 for obj in clazz.objects.filter(**kwargs):
>
>                     logger.info("    FOUND:'%s'" % (obj))
>
>                 vars(self)[dictname][key] =
> clazz.objects.filter(**kwargs).first()
>
>
>
>         return vars(self)[dictname][key]
>
>
>
> I see a lot of queries returning "get() returned more than one Target_File
> -- it returned 2". Please see the log attached. Unfortunately there was
> another error and bitbake still returned error code.
>
>
>
> I am interested to know what could be the reason for this error? Can it
> happen if a user starts a new build without first getting completed using
> same Toaster databases? How can I reset the database entries? Do I need to
> completely remove mysql database or there is something I can do within
> current build directory?
>
>
>
> Best Regards,
>
> Shakeel
>
>
>
> On Mon, Aug 27, 2018 at 2:14 PM Reyna, David <[email protected]>
> wrote:
>
> Hi Shakeel,
>
>
>
> 1) “WARNING: Could not add dependency to the package libc6 because
> ld-linux.so.2 is an unknown package”
>
>
>
> Yes, I have seen those messages. I have not debugged them deeply since the
> errors are largely passive, but it have been on my list of things to look
> at because there may be some details in the build data that we could be
> missing out.
>
>
>
> 2) From the looks of the trace, it appears that a found target file that
> has _*two*_ registered parents, at least using the cache/key mechanism of
> ‘*_cached_get’.*
>
>
>
> I have not myself seen this error.
>
>
>
> Since you have a reproducer, here is some sample code that you could in
> place of the existing “_cached_get” in
> “bitbake/lib/bb/ui/buildinfohelper.py”. It will:
>
>   (a) catch the exception,
>
>   (b) return the first found value so that the build can continue, and
>
>   (c) send to the log the value(s) that broke it.
>
>
>
> If you could insert this code (once it compiles), then we will have a
> better idea what happened.
>
>
>
> *    def _cached_get(self, clazz, **kwargs):*
>
> *        """ This is a memory-cached get. We assume that the objects will
> not change  in the database between gets.*
>
> *        """*
>
> *        assert issubclass(clazz, models.Model), "_cached_get needs to get
> the class as first argument"*
>
>
>
> *        key = ORMWrapper._build_key(**kwargs)*
>
> *        dictname = "objects_%s" % clazz.__name__*
>
>
>
> *        if not dictname in vars(self).keys():*
>
> *            vars(self)[dictname] = {}*
>
>
>
> *        if not key in vars(self)[dictname].keys():*
>
> *            try:*
>
> *                vars(self)[dictname][key] = clazz.objects.get(**kwargs)*
>
> *            except KeyError as e:*
>
> *                logger.debug(1, "ERROR:buildinfohelper:_cached_get: '%s'
> (%s)(%s)" % (e,key,**kwargs))*
>
> *                for obj in clazz.objects.filter(**kwargs):*
>
> *                    logger.debug(1, "    FOUND:'%s'" % (obj))*
>
> *                vars(self)[dictname][key] =
> clazz.objects.filter(**kwargs)[0]*
>
>
>
> *        return vars(self)[dictname][key]*
>
>
>
> BTW: this is the call in the trace that led to the error:
>
>             parent_obj = self._cached_get(Target_File, target =
> target_obj, path = parent_path, inodetype = Target_File.ITYPE_DIRECTORY)
>
>
>
> The second thing we could try is to directly look in the database and scan
> for duplicates for the above values. Any duplicates would cause the
> observed crash.
>
>
>
> Thanks,
>
> David
>
>
>
>
>
> *From:* [email protected] [
> mailto:[email protected] <[email protected]>]
> *On Behalf Of *Muhammad Shakeel Khan
> *Sent:* Monday, August 27, 2018 1:18 AM
> *To:* [email protected]
> *Subject:* [Toaster] Toaster Error - Rocko
>
>
>
> Hi,
>
>
>
> We are running the Toaster production instance based on Rocko (2.4.3) and
> seeing a lot of warnings when the build is completed. Our setup is:
>
>
>
> 1) Start Toaster production instance in isolation (toaster user) with
> nobuild option, just to monitor command line builds
>
> 2) A different user then fetches 'poky' separately and then sets up the
> build environment
>
> 3) Start toaster: $ source toaster start noweb
>
> 4) Run build: $ bitbake core-image-minimal
>
> 5) Stop toaster: $ source toaster stop
>
>
>
> This result in following warnings/notes when the build is completed
> (complete log attached)
>
>
>
> WARNING: Could not add dependency to the package libc6 because
> ld-linux.so.2 is an unknown package
>
> NOTE: We did not find one recipe for theconfiguration data package libc6
> Recipe matching query does not exist.
>
> WARNING: Could not add dependency to the package libc6 because
> ld-linux.so.2 is an unknown package
>
>
>
> What might be going wrong here? The Toaster web UI shows 100% progress.
>
>
>
> Also in one other custom image build we see the following error at the end
> of the build:
>
>
>
> ERROR: get() returned more than one Target_File -- it returned 2!
>
> Traceback (most recent call last):
>
>   File
> "/home/mshakeel/build-areas/build-area-rocko/sources/poky/bitbake/lib/bb/ui/toasterui.py",
> line 416, in main
>
>     buildinfohelper.store_target_package_data(event)
>
>   File
> "/home/mshakeel/build-areas/build-area-rocko/sources/poky/bitbake/lib/bb/ui/buildinfohelper.py",
> line 1383, in store_target_package_data
>
>
> self.orm_wrapper.save_target_file_information(self.internal_state['build'],
> target, filedata)
>
>   File
> "/home/mshakeel/build-areas/build-area-rocko/sources/poky/bitbake/lib/bb/ui/buildinfohelper.py",
> line 536, in save_target_file_information
>
>     parent_obj = self._cached_get(Target_File, target = target_obj, path =
> parent_path, inodetype = Target_File.ITYPE_DIRECTORY)
>
>   File
> "/home/mshakeel/build-areas/build-area-rocko/sources/poky/bitbake/lib/bb/ui/buildinfohelper.py",
> line 123, in _cached_get
>
>     vars(self)[dictname][key] = clazz.objects.get(**kwargs)
>
>   File
> "/home/mshakeel/.local/lib/python3.6/site-packages/django/db/models/manager.py",
> line 85, in manager_method
>
>     return getattr(self.get_queryset(), name)(*args, **kwargs)
>
>   File
> "/home/mshakeel/.local/lib/python3.6/site-packages/django/db/models/query.py",
> line 384, in get
>
>     (self.model._meta.object_name, num)
>
> orm.models.MultipleObjectsReturned: get() returned more than one
> Target_File -- it returned 2!
>
>
>
> NOTE: No commit since BUILDHISTORY_COMMIT != '1'
>
> NOTE: Build exited with errorcode 0
>
> NOTE: ToasterUI build done, brbe: None
>
> WARNING: Return value is 1
>
>
>
> Any thoughts about this error? We are using mysql
> (django.db.backends.mysql) for Toaster database.
>
>
>
> Best Regards,
>
> Shakeel
>
>
>
>