Re: Proposal: New procedures for handling packages in a GHC tree

On 06/11/12 02:56, Manuel M T Chakravarty wrote:

> I wonder about the loss of changes that both you and Ian mentioned.
> We are using git submodules for Accelerate, and yes, you need to use
> them in the right way (as you do with the nested repos), but I don't
> think we ever lost anything due to submodules. How does that happen?

Submodules are normally in a detached-HEAD state, and when you do 'git 
submodule update' it does a checkout to the new revision, so any commits 
you previously made are lost (well, they're still in the repo, but you 
have to dig into the reflog to get them back, and a future git gc will 
kill them):

I discovered this when I investigated submodules a while ago: 
http://hackage.haskell.org/trac/ghc/wiki/DarcsConversion#Theperspectiveonsubmodules

and this blog post is a good summary of the problems that can trip 
people up (see in particular the section "Oh were you using that?"):

http://codingkilledthecat.wordpress.com/2012/04/28/why-your-company-shouldnt-use-git-submodules/

The upshot is that there are new gotchas that people will run into with 
submodules, so we'll need some clear instructions on what to do, and 
perhaps some tooling (in sync-all) to catch the common mistakes.

> I also imagine that we might still like to keep the ./sync-all script to automate the use of submodules for the common use cases. (That should minimise the risk of mistakes.)

Right, the plan Ian had in mind was for sync-all pull to do an automatic 
submodule update too.

Cheers,
	Simon


> Manuel
>
>
> Simon Marlow <[email protected]>:
>> On 05/11/2012 10:52, Manuel M T Chakravarty wrote:
>>
>>> I agree with Geoff, submodules bring a certain complexity, but also
>>> advantages. We already pay for most complexities if we use submodules
>>> at all (such as having to use "git submodule" and remember to keep
>>> them in sync). If we pay already, why not reap the advantages for the
>>> rest of the repositories as well.
>>>
>>> For example, I often make changes in the main repo and the dph repo
>>> at the same time. It would be much more convenient if a branch would
>>> reflect the shared state of both (i.e., if dph would be a submodule
>>> of the main repo).
>>>
>>> In other words, if we use submodules at all (which is an idea I
>>> like), why add additional complexity by using different mechanisms
>>> for different subrepos? And why not reap the benefits of submodules
>>> across the board?
>>
>> I wouldn't object to trying that, if that's what people want.  I buy the advantages of submodules, and I buy the argument that since we have some submodules, it's probably easier to make them all submodules.
>>
>> I've been deliberately conservative in our adoption of submodules, because my own experience wasn't good when I tried them - it's easy to get very confused, and to lose changes.  The setup with separate repos has its disadvantages, but at least we fully understand it, and it's under our control.
>>
>> We're forced to use submodules for upstream packages, because we will sometimes need to move from tracking one branch to another branch.  This doesn't work well with separate repos, because GHC devs would need a git reset when we switch to tracking a different branch.  Hence the move to submodules.
>>
>>
>> Cheers,
>> 	Simon
>>
>>
>>
>>
>>> Manuel
>>>
>>>
>>> Geoffrey Mainland <[email protected]>:
>>>> On 11/02/2012 08:10 PM, Ian Lynagh wrote:
>>>>> Hi all,
>>>>>
>>>>> We have been working on a new way in which we handle the repositories
>>>>> that make up a GHC tree, in order to makes the process smoother for all
>>>>> parties.
>>>>>
>>>>> Our proposal is here:
>>>>>    http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions/Repositories
>>>>>
>>>>> Please have a look and let us know what you think, and especially
>>>>> whether you think there is anything that we've failed to take into
>>>>> account.
>>>>>
>>>>> Thanks
>>>>> Ian
>>>>
>>>> Hi Ian,
>>>>
>>>> Thanks for putting together the proposal.
>>>>
>>>> If we will now be using git submodules for external libraries, can we
>>>> not just use submodules for all libraries? That way the GHC repo hash
>>>> will uniquely specify everything needed to reproduce a build. Right now
>>>> it is painful to do any sort of bisect operation---doing so requires
>>>> digging through build logs to find fingerprint files for past builds.
>>>>
>>>> It looks like the proposal will leave us with the pain of fingerprints
>>>> and add the additional pain of git submodules. I am very much in favor
>>>> of just using submodules for everything.
>>>>
>>>> Geoff
>>>>
>>>> _______________________________________________
>>>> Cvs-ghc mailing list
>>>> [email protected]
>>>> http://www.haskell.org/mailman/listinfo/cvs-ghc
>>>
>>>
>>> _______________________________________________
>>> Cvs-ghc mailing list
>>> [email protected]
>>> http://www.haskell.org/mailman/listinfo/cvs-ghc
>>>
>>
>>
>> _______________________________________________
>> Cvs-ghc mailing list
>> [email protected]
>> http://www.haskell.org/mailman/listinfo/cvs-ghc
>
>
> _______________________________________________
> Cvs-ghc mailing list
> [email protected]
> http://www.haskell.org/mailman/listinfo/cvs-ghc
>