Re: ARM - events lost and cmpxchg

Hi Gunnar,

* Gunnar Larisch ([email protected]) wrote:
> Hi,
> 
> on ARM I noticed significant loss of events. At the
> end of a trace I get
> a printk message of over 100 lost events per second:
> 
> LTT : cpu_ : 122 events lost in cpu_ channel (cpu 0).
> 
> I have traced them to the place in ltt/ltt-relay.c
> where the check
> "*tsc == 0" is located. Is it normal that there are so
> much events lost?
> Which events are lost here?
> 

Those are events that are either :
- IRQs nested on top of an xtime update take has the xtime lock (on the same
  CPU).
- any execution context running on a different CPU from the one which is doing
  an xtime update, but the number of loops used to try taking the xtime lock
  would be too low compared to the total duration of the time xtime write
  lock in taken.

You might want to increment the number of loops in asm-arm/ltt.h. If it does not
solve your "problem", then you can consider that it is caused by IRQs nested on
top of xtime_lock being grabbed on the same CPU. The only solution would be to
disable IRQs at the same time this lock is taken. And if it does not solve your
problem, then maybe are there NMIs traced in your instrumentation ? (AFAIK, ARM
does not have NMIs).


> To get it compile clean, I had to add the function
> cmpxchg, which was
> missing on arm, needed in ltt/ltt-relay.c. I don't
> know, if it is
> possible to avoid locking interrupts on arm:
> 
> static inline long cmpxchg(atomic_long_t *v, long old,
> long new)
> {
>        long ret;
>        unsigned long flags;
> 
>        raw_local_irq_save(flags);
>        ret = v->counter;
>        if (likely(ret == old))
>                v->counter = new;
>        raw_local_irq_restore(flags);
> 
>        return ret;
> }
> 
> For this to work I also adapted the function
> atomic_long_cmpxchg
> in include/asm-generic/atomic.h. This will propably
> break it for other
> architectures, but I think "(l)->counter" should be
> placed in the
> architecture specific files.
> 
>  #define atomic_long_cmpxchg(l, old, new) \
> -       ((long)cmpxchg(&((l)->counter), (old), (new)))
> +       ((long)cmpxchg(l, (old), (new)))
> 
> Any suggestions?
> 

Sorry, I didn't notice that ARM does not define a cmpxchg in system.h.

Try the following fix (will be in lttng 0.6.44) :


--- a/include/asm-generic/atomic.h
+++ b/include/asm-generic/atomic.h
@@ -136,6 +136,20 @@ static inline long atomic_long_inc_not_z
        return (long)atomic64_inc_not_zero(v);
 }
 
+static inline long atomic_long_cmpxchg(atomic_long_t *l, long old, long new)
+{
+       atomic64_t *v = (atomic64_t *)l;
+
+       return (long)atomic64_cmpxchg(v, old, new);
+}
+
+static inline void atomic_long_xchg(atomic_long_t *l, long new)
+{
+       atomic64_t *v = (atomic64_t *)l;
+
+       atomic64_xchg(v, new);
+}
+
 #else
 
 typedef atomic_t atomic_long_t;
@@ -253,10 +267,20 @@ static inline long atomic_long_inc_not_z
        return (long)atomic_inc_not_zero(v);
 }
 
-#endif
+static inline long atomic_long_cmpxchg(atomic_long_t *l, long old, long new)
+{
+       atomic_t *v = (atomic_t *)l;
+
+       return (long)atomic_cmpxchg(v, old, new);
+}
+
+static inline void atomic_long_xchg(atomic_long_t *l, long new)
+{
+       atomic_t *v = (atomic_t *)l;
+
+       atomic_xchg(v, new);
+}
 
-#define atomic_long_cmpxchg(l, old, new) \
-       ((long)cmpxchg(&((l)->counter), (old), (new)))
-#define atomic_long_xchg(l, new) (xchg(&((l)->counter), (new)))
+#endif
 

Regards,

Mathieu


> Thanks,
> Gunnar
> 
> 
> 
> 
> 	
> 		
> ___________________________________________________________ 
> Der frühe Vogel fängt den Wurm. Hier gelangen Sie zum neuen Yahoo! Mail: http://mail.yahoo.de
> _______________________________________________
> Ltt-dev mailing list
> [email protected]
> http://listserv.shafik.org/mailman/listinfo/ltt-dev
> 
OpenPGP public key:              http://krystal.dyndns.org:8080/key/compudj.gpg
Key fingerprint:     8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68